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Overall Findings arid Recdmmehdatibns 

Our examination focused on two types of mbtivacional plans for 'teachers : 
merit pay plans (in which at least part of a teacher's pay is i?';nked-to per- 
fbrinarice) an4 nonmonetary "i^'erforSance-by-objectives" plans (in which per- 
fdrinance targets are set for individual teachers at the beginning of tRe year- 
arid, achievement against those targets is' subsequently monitored)^ <f 

The following are our major overall findings Detailed findihgjs on 
iridividuai issues are inaiuded throughout the vari.ous chapters of the repbrt.- 

- _ ^'..^^^^^^^^^^^y introducing teacher incentive plans, whether monetary or 
non-monetary; is a C9mplex and difficult undertaking. The many elements in- 
yol\^ed in a plan's design^ and implementation all need to be handled well or 
the plan can become of litcJfe use — or even counterproductive. (Detailed sug- . 
gestiohs on the design and impl^ementatidri of incentive plans for teachers are 
provided ^below.)- ' . - 

2 • The irif braatibri available f rbm past research^ Sjchbbl district 
experiences, arid the evaluatibri literature prbvides little cbnyiricirig evidence 
(that is* eviderice v/ith everi toiriimal rigor)--brie way pr the other-^n whether 
%j|acherjjnerit pay br perf brmance-by-bb jectives plans have substantially 
affected student achievement ^ teacher reteritibri rates^ br the ability to 
attract new quality teachers. This^ hbwever^ does not mean that these plans* 
have ribt had significant ^rapacts in these areas, only that no convincing 
eviderice has yet been developed. There are indications that sonje plans have 
produced improvements^ but the districts do not hav$ sufficient information to 
provide convincing suppport jof those claims i Only a few of the -school 
Jistricts have conducted formal evaluations of their plans, and even these 
have suffered ftom a lack of pre-plan data and from the absence of any 
consideration of plausible alternate explanations for observed imprdvetnents 
(such as economic conditions that could explain-high teacher retention rates 
in recent years). And we fbutid no multi^site evaluations that had attempted 
to exatiiiae more than one school district in a systematic way that cbuld * 
provide generalizable evidence as to what plan characteristics -arid whab 
conditions seem associated with the success, or failure, of these pl^ris. 

Overall Findings arid Recbmijerida tibris bri Monetary Iriceritive Plaris 

_ » - . 

3. The limited irif brma4:ibri ribw available strongly suggests that suc- 
cessfully iritrbducirig pay-f br-perf brmarice plaris arid sustairiing them effec- 
tively is cbmplex arid dif f icult^^ requiririg that mariys elements be done 
prbperly. Exhibit lin Chapter 1 lists the mariy issues that a school district 
needs to corisrider. If any orie elemerit is dbrie pbbrly br breaks down^ the 
whole process can soair. Especially impbrtant requiremerits f or successful 
merit pay plans appear to be the fallowing: (a) teacher participation aud 
cooperation, o^ at least the absencje of substantial opposition^ (b)*a teacher 
evaluation process that p^artici pant's perce^ive to be reasonably fair and 
objective, and (c) the ability to provide .significant awards (e.g. $1,000 "per 
year or more) to ail teachers- who -deserve them, even if this is a substantial 
proportion of the teachersi ' , 

4i The evidence that is available indicates that a number of character- 
istics are associated with positive consequences for' monetary incentive plans^ 



or, convfcirseiy, that the absence of these characteriistics is often associated 
with substantial negstive* consequenees f or trie plans. These lead t:d the 
.following suggestidns«f or school distlricts. ^_ - 

d Leave • ample t'itie to diesign arid iinpliameht the ^lari/ iricljJdirig (a<) time 
to ^gairi teacher participatibri in its desighj (b) time to develbf) 
apprbpriate teacher performance evaluation procedures^ arid, (c) time* 
t'b trairi persbrfnel in plan prbcedures. (Chapter 6) 

6 Provide significant bonuses ($1^000 or greater) and allow them to be 
given to a substantial propoptibri of the teachers that perform meri- 
toriously. If at all possible, use no fixed quota at all.; ThisNiili 
reduce the iik.>alihood of complaints of unfairness and favoritism ajtd 
of discouraging staff that performed well bat did not squeeze in 
under the quotas if the school district cannot be assured of the 
annual availability of adequate funds for such merit .awards , it 
should not implement a merit pay plan but should instead consider 
nor.-Tnonetsry incentive plans (discussed" later) . Problems caused by 
tight quotas can be severe arid can undermine otherwise sound merit 
pay plans. (Chapter 3) ' ^ ■ 

d Use one-year bonuses rather than permarierit salary increases as awards 
in brder .to link, the added cdmperisatidri to the iperidd of meritdridus 
perf drmarice. ( Some Idrig-lived plans , hdwevetj, do use salary 
increases as their fbrm bf award.) (Chapter 3) , . * 

o Ensure substantive participatibri by teachers in the cfesigri^ implemeri- 
tatibn^ and monitoring of the plan.' This is often recommended by re- 
searchers ana .school of ficials alike i but it is frequently violate;^' 
by school districts wanting to move quickly to introduce a merit pay 
plan. Initial opposition from teacher associations should be ex- ^ 
pected, and in some districts formal participation may be difficult 
to achieve. if opposition cannot be overcome^ and teacher willingness : 
to try the plan (even if they do not actively support it) canncft be 
obtained^ it is likely to be better to develop a non-monetary plani 
(Chapter 6) " _ . 

. iP 

o When usj^ng objective teacher evaluation methods such as student test 
scores: (a) make explicit provisidn in the procedure for considering 
"external" factdrs such a^v differerices amdng classes is student 
ability and mbtivatidn, and (b) uridertake efforts to develop tests 
that better matc^ the schddl district's dwn curriculum^ Both of 
these suggestibrii are difficult tb implement , but satisfactory 
answers are needed if such evaluatibri methods are td be viewed^ as an 
equitable basis fbr merit pay decisibris . Hbustbri^s apprdach td the 
fi-rst issue (by using statistical regression arialysis tb adjust for 
differences in class characteristics ) is one pbssible technique ^ but 
one that needs* more development and testing. (It was rec*eritiy 
discontinued in Houston as being too- difficult for te^achers to 
understand.) (Chapter 5) • 



)r annual ot biennial ^valuation of the plan against the 
>trict's ob jectives to determine whether^ it cbntinu^ tb 



Providl for 

school district's objectives to^determine whether it continue^ to be 
cost-effective. Take corrective action if itf is found tb be 
defective^ (Chapter 8) 



5. Wfc found some unusual variations in monetary plans that appeared to 
B associated with sutcess. Other schooi districts should consider these: 



(a) Offer' a penu of' awards, with cash bonuses being but one possibility 
(as is being tried in the Cataline 'Fodthiils and Tempe Union High 
School Districts, both in* Arizona) In such a plan, the teacher 

.chooses whether to take the award in cash, have the district pay for 
attejidance at^cbnferences pr workshops, cTr have the district support 

special classro^ins tructiohal/^nrichmeht fTrdjects such as the 

purchase of a iicro-computer T telescope , aquarium, or camera i This 
type of plan may particularly appeal to the professionalism of 
teachets. (Chapter 3) ^ . * 

(b) Makfe the merit plan voluntary. That is^ each year before the awards' 
are announced , have teachers indicate formally whether or not they 

^ want to. be considered for merit pay. This approach was associated 
with a number of plans that appeared successful or at least that met 
U-elatively little opposition from teachers. 'The requirement that a 
1 teacher volunteer in order to be considered for a merit award means, 
that a teacher can avoid the embarrassment "^of not receiving an 
award, thereby avoiding what can be a major morale proble^. 
(Chapter 6) f * - 

(c) include a "group" incentive cdmpdnent in the plan. That is^ give^at 
least part of the awards to groups of teachers based on the group s 
perforaance, as in Houston, Round Valley (Calif ornia), and Selling 

«^ (Oklahoma). This has the. advantage . of encouraging cooperation . 

within a school district and avoiding the criticism (especially - 
plans with restrictive quotas) that tnerit pay will cause divisive-- 
ness. ^The group can be -a school (as in Houston and Sailing), spe- 
cific grade levels (as in Round \Jalley), or perhaps those teaching 
particular subject matter (but covering many grade levels). The best 
approach is likely to be a combination of individiial and group 
incentives, as was done in the three plans jaen^tiofied above. 
(Chapter 4) ' / 

"* 

(d) Use perfbrmance-by-objectives procedures aS a way to evaluate 
employee perfonnanbe for merit pay plans'. This •approach entails 
considerable participation b> individoal teachers in a highly 
professional way. If adequate staff training-'is provided in setting 
objectives and targets, and adequate proceauresare developed to 
ieasure target achievement, this jiroceas can provide a/reasonably 
objective evaluation approach, one-'fhat""^fe«^les the performance 
evaluation to b% tailored to z^-e character i^J.n^ividual schools, 

")grades, subjects^ or classes. (There are a nbrobfer of pitfalls . with 
^this procedure, h-.wcverj see Chapter^; 5 and 7.) The approach us|d 
by Round Vallev v'talif brhia) — to have a committee rate a teacher s 
targets at the' beginning. of theyear as 'to relative difficulty and 
importance to the school district— is a way to provide more com- 
parable and thus fairer teacher evaluations -under this : technique . 
(Chapters 5 and 7) ' " ■ . ^ 



^ ■ ; . • ' ■ • .r . : : / 

b. The key technical issue in incentive plans i^ how » ©3 evaluate tead^er 
pfcrfbraahce. There art substantial differences >of' . opinion. % Despite the ' 
considerable wort; done on teacher evaluation over the years, we louTtd little 
research that, adequately validated any of the ma^jor approaches— saperv±§i3ry 
ratings^ peer ratings^ student -f^tiiigs , achievement of- pre-set targerts^ parent 
ratings i and even student test scores^ — as indicators of teacher ability to 
improve student learning or studen? affecCiv^ outcomes (such as self-esteem, 
relations with others, ^aiues, etci)i -There has been considerable research on-^ 
the relationship of various teacher characteristics to 'student achievements 
This research is also relevant to the question of the usefulness of supervi- 
sory and peer ratings of teachers for* monetary incentive plans.. The iafbrma-i' 
tion we reviewed lea-^es considerable ddiibt about th^ existence of substantive 
-li-nks between specific' teaching characteristics-C including teacher practices 
arid be?iavidr) 'arid studerit achievem^rit. ^Cbrisixierably more research appears 
riecessary to cimpare teacher evaluation approaches arid to estimate what^eacti 
can and canript db—bpth individually arid in combination. For the present^ 
school di^stri^^ts ViH probably warit to use a combination of teacher evaluation 
r uacnes . (Chapter 

7. A ma^qr problem for school districts at preset is the inadequacy of 
student achievement tests for helping to evaluate teacK^r performance. Cur- 
Gently such tests often .provide, inadequate coverage of material that, school 

systems want teachers to teach^ There are deficiencies as to <coritent^ sub- 

-.- — - - — — 

jects covered^ grade levels covered, and th^ 'coverage of both f acC-testing a^nd 
higher-order learning (eigi creative thinking); Thus, armajor n^ed for inceji- 
tive programs is the development of comprehensive tests of studeflt progress 
that better match the local school system's curriculum^ As 'th^e become ^ 
available, th<^ will permit school districts to better use studerit testing fbr 
teacher perf prmarice evaluation. This pPirit was stresied/by districts that we 
talked witt^ whbSe merit pay systems irivblved testing. With more cbmpreherislve' 
test coverage I the testirig shbuld be able to provide mbre guidarice to teachers 
arid schools cbricerriirig ^learriirig deficiericies arid achievemerits. (Chapter 5) 

8» A fce_5^ related is^ue. is whether tiie performance ev^luatibri procedures 
used to determine merit pay eligibility can also provide informatioh fbr 
guiding teacher improvement. Unfortunately, the prbted^ures that s^em most 
likely to be abl.e to provide specific suggestions for teacher impfoy^ement are 
supervisory and peer' ratings^ which alio appear to be the most _su£^ect4^^the 
least objective, a.nd the noit questionable in terms ef validit^^^n ^the^^pse 
of being significantly related ""^^to student achievement) i The development ^5d. 
use of 3^adertt tests that better match the curriculum of tne local ^school't''^':/^ 
district may well permit the district to use those tests both to evaluate 
teacher performance and* to pin-point subject tsiatter where th^e teaching^ process 
..needs strengthenings (Chapter 5) • . \ 

. - ■ ■ ■ ■ " ' 

9* ^Career gadder/master teacher plaris are of irit;erest throughout the ' 
United St^ates, arid 'numerous variations -^are appearing.' At present, thecte 
appears tb be eveti less itifbrmatibri about the success bf these plaris. • «Mb6t 
trials are^ just "begiririirig, arid the riumber of variatibris is leadirig tb sbme 
cbriifusibr. Those variatibris built ardurid riew jbb duties fbr teachers— such as 
te^hing other teachers arid develbpirig curriculum—prpbably. db riot ^ fall urider 
the usual definitions bf incentive programs that are aimed at ericburagirig im- 
proved perfbrmance in bne's preserit jbb. ThiTs^ plaris that mbve teicheM _away 
from teachirig students as their majbr jbtj '^activity represent quite a differerit 



10 



i • .. ■ 

approach, than merit pay "and hbn-ri;brit:tary -per f brmance-.by-ob jectiv^ plattis; And 
they probably j^rdvide at any given tim^ opportunities for only a sinall prbpbi:^^ 
tiph'^bf teachers. Qh the bther hand ^ some career deveibpment plans are. a 
liybriii. * They combine tenures bf bbth merit pay and career ladder plans i 
prbvi ling higher teacher pay grades (and higher status/ fbr a cbmbinatibh of 
criteria such as perfbrmajice ratings^ participation in special district 
pfoje'cti^Si educational credit's ^ and lbngevity--but with teaching^ tudents still 
being the teach^t's main- job. Our findings on merit pay plans also apply to 
these hybrid plans (Chapter 9) * > , . • ' 



^ 



Findings and Recommen d ations on Non-Monet a ry' Perf brmancg-by-Qb ject i v es- 
(Chapter 7) . ^ 

^ , _ _ 

10. Non-monetary perf ormance-by-bbjectives (PBO) pVbgrams have potential 

fbr encbiiraging improvements iti teacher performance at relatively low cost to 

s-chbol districts whilie avoiding the hassles of tea associated with monetary 
inceriafeve plans. TPiis bptibn should l)e explored further By school districts; . 

- . . « _ _ _ " . _ i • . ^ J. ' . .. .... 

11. The five PBO programs fbr teachers that we exajniried .did tibt appear 
tQ explbit the 'pbtehtial of PBO to mbtivate ..teachers to perfbrin better* 
Although these programs gave some limited evidence bf imprbv'ing teacher 
performance i their motivational effectiveness was cbmprbmisjed by inattehtibh 
to, ^ number of important design and implemecitation Issues. 

^ . _■ ^ _ ^ ^ . , - - - ■ " 

12. Nevertheless, evidence from experience and' research with PBO for 

non-teaching personnel in: the public -and private sectors suggests that PBO^ 
programs for teachers can stimulate improved performance ii properly designed^ 
implemented^ and maintained i . 

13* We sujjgest that p^rfonnance-by-objectives programs 6e tried by 
school districts .as a motivator of teaching personneii These trials.^ however, 
shbuld attempt to oprrect the deficiencies identified in past programs i This 
means trying PBO programs that: \ ' ^ 

_ __ _ ; _ . ■ . » . 

^ Cbver a^lj br at Inast most, staff (eig; not only non-tenured and 
other prbfeatibhary staff but tenured teachers as well). * 
/ I .._ _ . 

- Emphasize the use bf butcbme-briehted targets (and/ perhaps, gbals 
involving the periodic completion bf special prbdtactlvity 
improvement projects). 

- Target the achievement of excellence rather than only nLinimum 
performance standards or specific performance prbblems. / . 

- Emphasize the specification .of objective criteria and data 
collection procedures by which Carget achievement can be measured. 

► **• 

^- - Provide central coordination and 'oversight of target quality and " 
y,.-' ^ difficulty. ^ ' , • 



Give recognition to staff with outstanding performance with respect 
to target achievements ^ 'L^ 
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- Empha'bize tbp-Ievel inanageneht support for the PBO effort arid take 
wha tevi^r • acidi t ibrial steps are needed to erisure that the PbO effort ; 

J X'l taicen seribus*ly. 

- Regularly "revitalize" the program to ensure employee uriderstaridirig 
and atto-ntibn* • 

- Use the iuf ormatibh on target achievement as a major part of the . 
annual teache'r evaluation process ^ and make it clear to teachers 

<^ what specific roie the achievement of targets wiii play in the" 
evaiaation*^ process • ^ 

L ons to State Governments (Chapter id) - • 

14. State goverrimerits should avoid nsandating highly specific require- 
merits relating tc teacher incentive plans (ii^cluding over-specifying^ teacher 

ev^luaricn requireu^er*!: s ) — at least until there is much better and more 
convincing evid*ehce chat certain practices work well arid urider a variety of 
local conditions. * - ' • 

_ . . ' " _ _ _ 

15. States , should encourage school districts to experimerit with teacher 

incentive plans but at the same time provide for aystematic evaluation of such' 
plans, using compatible Evaluation procedures so chat the experiences of these 
pilot districts can -lat'eY be used to provide guidance to other districts. 



Recommendations for N eeded N ational Reaear ch 



16i The OiSi Department of Education^ state departments of education, 
and/or foundations sl>gur<i sponsor systema^tic trialji by school districts of a 
variety -of teacher motivational jjlans. These should b^ accompanied by careful 
evaluaticm. Pliris' should be tested in a variety of settings — large and spall, 
"wealthy" arid "pbbf" districts. 



- ft. ? ' 

Plan variations that appear highly desirable to inj^lude in such trials 

are': ^ ,1 

o Trials t;hat involve a merita bf possible awards , . iricludlrig riot orily 

monetary bonuses but also such optibris as atteridarice at. prbfessibrial 
conferences or wbrkshbps and special purchase bf _ classrbbm iristrtac- 
tional/enrichment materials (similar tb the Cat'aliria Foothill^ School 
District ^program in Tucson^ Arizbha). ^ 

o Trials that involve .performance by, objectives as a way tia assess 
individual teacher performances ^fse trials should preferably 
include both monetary plans (such as Round Valley, Ca4±fornia5 and 
riOri-morietary^ plans (such as Salt take eity)i However^ to the extent 
possible, ^these trials should avoid the basic deficiencies noted in 
'- X:hapter 7 arid should incorporate the characteristics listed in #13 

abbve. ^ ; - 

' b Trials that^fbctis bri cbmfiiri'atibris of teacher evaluatTSn procedures ^ 
iheludirig student test scores, student (arid perhaps parent^- ratings. 
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target achievenBeut , as weii as systematic supef'visory ratiqgs 
(including tnore structured classroom observations such as the 
procedures being developed in Florida )i ♦ - ^ - 

Rigbrpus evaluation designs* (such as controlled evaluations using random ^ 
assignment) are probably not feasible. (Cdntrol ling the trials would be very 
difficulty and' rahdbmizatidn involving teachers would probably be iiif easible 
for monetary plans.) Nevertheless^ sites should be carefully evaluated using 
the same data cbllectibh procedures and covering such evaluation criteria as: 

o the costs of the plahs-^ administrative as well Ss award cbstS| staff 
time costs as well as but-of-pocicet cbsts); 

o Effects o^ student learning (preferably including both cognitive and 
affective elements) using a variety of approaches— student test 
results^ student and parent feedback, supervisory ratings, etc.; 

* _ ■ 

o Effects on, teaching practices (e.g. ^ic they change because of the 
plan and. in what way?); 

d Effects on teacher morale and attitudes; and 

d Effects dn teacher retention and ability to attract "quality" 
teachers » 

Particular attention should be given in these evaluations to obtaining 

baseline data bh the various evaluative criteria (iie; data relating to 

peribds befbre the hew plan is implemented)-- -including baseline student scores 
and student^ parent^ and teacher ratings and attitudes. 

Two basic types bf evaluations will prdbably be necessary: (1) "after- 
the-fact" evaluations of plans that have been in operation two or more years 
(this length bf time is needed to assure that the plans have beefi "shaken 
down" and that the districts have some reasbnable amount of experience with 
the plan's bperatibh)^ and (2) evaluatibds that begin pridj; to the implementa- 
tion of th^ plan and fbllbw the plan's operatibh fbr sbtne_ period of time. The 
first approach has the advantage bf being easier and can be ddne relatively 
quickly, but it depends on the existence bf trials bf the types of plans of 
interest and the availability bf at least minimal pre-plan "baseline" data, 
the second approach has the advantage that the baseline data shbxiLd be supe- 
rior; (It does ^ot have to depend on reconstructing baseline evaluative data^ 
and information such as baseline student and teacher attitudes can be readily 
collected.) But the second method will require a minimum bf 2-3 years to 
follow the plans long enough to provide meaningful evaluative data bh impacts. 

17. More work is needed to develop^ student tests that better match the • - 
curriculum df individual schoofc districts and cover ail s^^^ grade 
levels. ThdUgh this is in part a local matter, it is unlikely to be efficient 
natidnally Jdr each school district to start from scratch in develbpihg such 
^ tests. The availability df ties ts that more adequately indicate the extent to 
which students are learning what the teachers ape trying to teach is a corner- 
stbhe tb adequate teacher evaluation and teacher incentive plans. Fortu- 
nately i-th±s-appe|-rs- to be a directidn *in which testing is already moving^ 

■ ■ : \ 
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iSi. Field research is neeaed to exaxint: and conpsre tne validity ana 
coverage of varloas teacher performance evaluation approaches, tnciodiag _ 
supervisory, student, and parent ratings of .teacher impacts, (both cognitive 
and affective) and of teacher classroom perf brniahce ; studeat test scores_^ 
(especially with more iocaily appropriate tests--as noted above) ; and perfor- 
mance-by-bbjectives techniques (these have the addea problem of the 
comparability of individual, targets that may reflect differing degrees of 
importance and difficulty); . ^ 

19. More attention seems needed on assessing a teacher's impact on 
affective outcomes for students (positive seif-attitudes ^ personal adjustment, 
reiationshipi witn others, self-reliance, etc.). We found little effort by 
school districts to use techniques such as parent and student feedback, for 
assessing changes in affective outcomes ,• nor did we find any relevant methb- 
doiogies presented in the teacher evalution literature that we examined. 
Though tnis is probably a iecbhdary item for the research agend'a, measurement 
of afrective outcomes may /be impbrtant'to some school districts.. 

20. Finaiiv, to help scnobi districts tnat use supervisory (or peer) _ 
ratingsof teachers as a major part of their incentive plan (or teacher 
evaluation process), the development of more "ahchbred" rating instruments 
seems badly needed. As discussed in Chapter->5, much more specific definitions 
and descriptibns are needed for each characteristic being rated ar^ for each 
grade bn the rating scale. This is needed to achieve reasonably re±±ab±e 
ratings that will permit different raters, rating different teachers and/or 
different performance periods, to provide reasonably comparable ratings. 



PART I 
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CHAPTER 1 



Introdaction and Scope 



I 

We report here oh a four-ionth examination of the issues involved in 
developing incentive plans for teachers. Our examination focused primarily on 
two types of plans: < 

o Tea cher merit p a y plan s^ These art defined here as a formal process 
in which a significant cimouat of a teacher's cdmpehsatidh is based on 
^^^^Pli^i^^^i^ substantive assessment of at least some aspect of 
teacher performance — and on at least ah ahhual basis. 

o Perf drma ace^by-bbjectives plahs . This is management by objective^ 
(MBO) as adapted for teachers. Here^ teachers and their supervisors 
jointly set bbjectiyes^ usually for the school year. Actual perfor- 
^ maiice is subsequently compared to the targeted performances This 
process cah be used as part of a merit pay plan, but it can also be ' 
used as a motivatibhal tool without being directly linked to teacher 
cbmpehsatibh. 

' _ _ _ 1 _ 

A third type of plah^ the career ladder/master teacher/differentiated 
Staff ihg approach is examihed very briefly as it relates to teacher incentive 
pjlihs. The^e terms are generally applied to plans in which new posit ions are 
established involving special tasks such as teaching other teachers. Higher 
pay is associated with these positions. However^ these do hot geherally focus 
on incentives*'' to improve teachers' performahpe ih teachihg studehts. 

exclude from bur definition -of ihcehtive plans those approaches that 
primarily prbvide additional pay for teachihg subjects in short supply (such 
as mathematics and g^iepce)^ for teachihg in particular schools that otherwise 
find it difficult to attract teachers^ or for extra education. 

Study Methodology 

We used two primary procedjires: examination of published (and sbine un- 
published) materials, ahd^elephone interviews with representatives of 22 
school districts supplemented by examination of materials on the plans in 
those districts^ (Reports oq 16 of these are included in Part II.) We alsb 



drew on the_ Urban Inscitiite-'s previous work on nidtlvatlbhal prdgrains for state 
and local government employees other than teachers and, to a lesser extent^ on 
the experiences of the private sector regarding motivational programs; With 

the highly . limited resources and ti^e available for this study^ we were not 



ablie to do any on-site data collection at the school districts, nor any sig- 
nificant amount of checking on information furnished to us. This is a major 
limitation for any examination of the practical problems facing school 
'districts. . " 

Our examinatidn of the published literature focused primarily ox^ merit 
pay plans. In addition^ we more selectively examined a number of related 
topics that bear directly on impbrtaht incentive plan issues^ including 
teacher evaluation^ research on the relationship between teaching characteris-* 
tics and evidence on student learnings motivation theory^ and student testing » 
Between the rapidly escalating amount of published material on teacher merit 
pay and the even greater amounts of material on these other topics,, there is a 
considerable amount of potentially relevant information. In a limited study 
such as this, it was not possible to cover all these topics cdmprehensively. 
Our findings, thus, are by no means definitive, but they do represent bur 
synthesis from a large amduiit of such materials • * 

We alsd had td be quite selective in dur telephone interviewing with 

schodl districts. There is a rapidly grdwing number of school districts at*^ 

- - - ■ : -- - : ' V . 

tempting some varlatlbh of these Ihcehtlve plans. With dur limited resdurces^ 

■ - ' . . * , - ... 

- - *" ■ , ' 

we fdcused primarily dh thdse plans that had been in existence for at least a 

year so that we could examine schools with considerable experience with the 

plans. We sought interviews from school districts that were reported to ha^re 

substantial incentive plans and not, for example, locations whose incentives 

. 17 



___ _^ _-r-l.. . _■• 

were aimed at only a very small proportion of the teachers or that appeared to 
represent only: minor steps forward from longevity-based approaches. We ihclu- 
ded in our examination a few districts that had trie4 plans . for a few years 
and then tertsiinated them--td provide a different perspective. Tbt- btivious 
reasons, adini til stra tors of locations having ah bn—golng plan are generally 
considerably more optimistic about incentive plans than those whcrse plans have 
been terminated.- . ' 

To select the school districts for our telephone interviews, we used re- 
ports on ihdividaal school districts from articles in prdfessldnal journals ^ 

I ___ _____ _ _ _ __ ._ _ \ 

newspaper articles, and recent national reports of irtceritivje plan experiences. 

Especially useful were ^he Educational Research Service's 1979 and 1983 

- -i -- - 

reports on teacher merit pay and M. Donald Thomas' 1979 book on performance- 

by-bb jectives plans. Our interviews are hot a rahddm sample (we were not 

seeking statistical data for thll report) but a selective sample based on our 

objective of examining a variety of substahtlve ihcehtive programs. Wi would 

have much preferred to examine mahy more thah the tvehty or so districts we 

interviewed ahd would have much preferred e^mining each in greater depth. 

There clearly is still a grea,t scarcity of descrip^tive and - morys importantly - 

evaluative ihformatibh on actual schbbl district experiences with thesie plans. 

Other Limitatibhs on Our Scope 



An important issue we have not attempted to examine is the extent to 



which teaching ^affects student leamingi This has b^en the subject of - 
considerable debate for many years^ A basic assumption isehind teacher 
incentive plans is that teachers can indeed have a non-trivial effect oh 
student lea mi ngi For this research, we have accepted this assumption. Some 
researchers such as Centra and Potter (1980) report that-- teacher effects are 
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We have attemped to lay. out the key issoes as tl^ey became apparent from the 
iiteratore and the experiences of school systems trying, such ^plans* 

We have part icalariy attempted to identify ^filrlcai evidence on the • 
impacts of these plans from the research literature and from evidence avail-- 
able from school districts that have Implemented these plans'* As stated by . 

■ _ ■ * ;_ __ _ _ _ _ • ^ 

Innan (1983), "the present educational pay system dbe,s hot differentiate be- 
tween goody bad» of indifferent teachers. For the most part, all teachers are 
treated the same • • • ."^ Without substantial; evidence to the contrary^ it 
cahhbt be proven that this is necessarily bad^ the burden is on others to 
provide proof that alter^batives such as teacher incentive plans can lead to 
^jbr improvements worth their costs. Tfie principle ''if it vorks, don't fix 

is an Important concern for • individual school districts. . However » in re- 
cent years there has bee^/growing national concern that the educational system 
is not working the way it shduld, and this concern escalated greatly in 1983* 
Now there is much talk about the need to fix the ^system. major question 
facing educational officials is which types of iticehtive 4>lah8| if ahyi can 

help fix the system and what* steps should be taken to Kelp assure that such 

^ _ ? _ 

plans will be effective* 

f ' ■ - ' 

Content of the Report ' 

the report is divided into two parts, the first part discusses the key 
issues involved in designing and implementing a plan* Each of Chapters 2-10 
discusses ^ different major issue, the issues this report covers are lis^d 
in Exhibit (School districts might want to use this exhibit to help them 

keep track of the major decisions they need to make in designing and implement- 
ing teacher incentive plans. ) Chapter 7 focuses solely on non-monetary versions 
of perf ormance-by-objectives programs. Chapter 9 notes the relation between 
teacher incentive plans and career ladder/master^ejich&r—plans^ j Our overall 
f indings and recommendations' are presented at the beginning of the report. ^ 



Part '^.I presents the case reports for a namber of school district 
"plans. Each report describes the plan and presents bur findings on any 
readily available evalxiative Itifbrmatlbri bti the . plan» ' Reports on tnerit pay 
plans are presented first (alphabetically) follbwed by the reports on 
perforiance-by-ob jectives (nbh-nnonetary ) plans • 



; Exhibit 1 . 

List of Jlajor Issaes For teacher In e ent 4^-g^ Claris 

. . __ ___ _ _ __ . ' : / , 

What should be the objectives of the plan? (Chapter 2) 

lii What type and size of rewards should be used? (Chapter 3) . 
What type of rewards should be used? _ 

How large should the awards be? Should every recipient receive the 

same size reward? 
What proportion of. teachers' compehsation^shburd be linked to ^ 

perfbrma^hce? Should increases be in addition to' cpS.t-of-living or 

other ac4:bss-the*board adjustments? 
How long should the performance period be?, / < , 

III. Whd shduld be eligible for awards? (Chapter 4) ; ^ _ 

Should individual or group incentives (or a c6mb± nation*^) he used? 

Should participation be voluntary? 

Should persons other than teachers be included? 

How many persons should be able to receive awards? Should there be 
some form of quota? > 

' \- - 

IV. - How should teacher performance be evaluated? (Chapter 5) 

What elements should be evaluated? 

What type of teacher performance evaluation procedures should be 
used? 

How should teacher evaluations be linked to specific award ambuhts? 

V. Other incentive plan design issues, (Chapter 6J _ _ 

To what extent can and should the teacher evaluation procedures also 
be used to identify ways to improve teacher performance? 

To what extent should information on receives the awards be 
promulgated? _ _ _ _ * 

To what extent shdiild teachers and teacher associatidns participate 
iti the design and Implementatldn of the plan? 

How much^adVance planning and preparation time is heeded? 

Should there be an appeal process? 

What preconditions are heeded fdr success? 

— - ^ ■- - / - ' " ' ■ 

VI. What are the Issues and potential kssdclated with non^mbnetary 

perf drmahce-by-dbjectives plans? (Chapter 7) 



VEIi Issues after the plan has been put Into operation (Chapter 8) 
Will adequate funds be made available each year? 

What provision for subsequent evaluation and revision of the plan 
/ should be made? ' 

Are there special activities and resources needed f or jOperating thje 
plan? 



VIII. . How should the Incentive plan be integrated with career ladder/master 
teacher plans? (Chapter 9) 

IX; - — What sh o uld b e— th^ role o^f t^ie State ^overnm 10) U ^ 



what Shbiild Be the Objectives of The Plan? 

Each school district introducing a merit p'ay plan will have its own 
particular objectives. The following five objectives are p^^e^alent and seem 
to j?€ the primary ones: - 

iw To imprbve instruction and thus improve student achievement* 

2. To improve the ability of the system to retain quality teachers. 

3. To improve the ability of, the school system to recruit quality 
teachers* V ^ * 

4. To make increased pay more acceptable i9^^i_°^^^^^?i?_^?§ to the*^ 
public I thus increasing their willingness to provide funding for 
education and extra cdmpens.atidn for teachers. 

5. To pay teacher? more fairly> based on the value of their 
cbhtributioiF. 



The basic assumption behind the first objective, itiiprdving instruction 

and thus student achievement^ is that these plans cat^ motivate teachers to 

alter their activities in ways that can improve student learning. " Objectives 

Nd. 2 and 3 assume that teachers will perceive the incentives as greater pp- 

pbrtuhities fdr a rewarding career, encouraging quality personnel to enter and 

^ ^ . - . - 

remain in the teaching field. The fourth objective ±s important to those be^ 

lievihg that teachers are underpaid and that incentive plans are one way to 
help correct this. The assumption is that litflcing at least part of a ^ 
teacher's pay to performance will make it easier to gain j^ubiic'^ support for 
increased teacher cbmpehsatioh. ^ * ' , 

Finally, objective No. 5 provides a ju^ification for monetary incentive 
plans even if there is no specific ihtehtibh that the plans motivate teachers 
to improve their performance. This objective i3 based bn the basic principle 
that employees that do better should earn more. This is a much less demanding 
objective than Noi 1 in the sense that it does not require that additional. pay 



ihdtivatis teachers to pfe'rfbrni better arid thus does riot require that butcbmes 

such as improved stuBerii achievetnerit occur Becausia of the incentive plani 

. _ _ _■ 

The degree of success of a plan in achieving the first four objectives 

can potentially be determined through an examination of empirical evidence i 

Pi ^ , ■ . • ■ 

No such support is required for the last objective, but even here there is. 
still the question as to whether the particular procedure for determining 
*which teachers have performed better is valid. 

School districts should try to be explicit about what they hope the pl.ari 
•ill accbnplish arid prbvide for p^ribdic feedback bri hbw well the plan is 
^achieving its objectives. We fburid that most • district^^^id nbt f brmally 
evaluate their plan's accomplishments^ even infrequently (see Chapter 8 for 

\' 

further discussion); 
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^ _ : « • eHAPTER 3 ' ( 

■ T : • ' • 

Wliat Type and Size Rewards Should Be Used?. % ^ 

io this chapter we discuss whtt types of;, rewards should be use<d^ how 
Isrge they should be, 'and how long the perfbiniiance period should be. 

Two types of plans are the primary focus of this report: plans tliat 
relate teacher compehsatibh to perfbrin^iic and those that *are nonmonetary but 
use some form of perfdriiiaoce'^b^r'-'o^jectives. xThe" advantages and disadvantages 
of the latter are discussed in Chapter ?• This chapter pertains only to 
monetary planp. ^ 
What Type of Rewards Shoaid Be Used ? ^ ' 

We found three forms of rewards in use: ( 1). peni^nent changes to 
teachers^ salaries, (2) bdniises over and above the teacher's . salary , and «(3) a 
"menu" of awards where the teacher ca.n clioose between taking the ayard in 
cashj attendance at a cooferehce^ or special teaching materials. 

1. Salary adjustmehys. Traditionally^ salary increases for public 
employees ha e been based primarily on longevity and educational credits. 



(This api^ies to both education and other local ai|d state goverxapeat person'- 
nel.) Employee performance has traditionally been used primarily to identify 
whether employees have not been performing adequately and thus are subject to 
sL probation period or discharge. 

Incentive plans^ tied to salary increases attempt to link such increases 

at least partially to performance. Examples of such plans include those of 

_ , _ • ^ 

Ladue (Missouri)^ Latce Forest (Illihbis)^ Bristol (Vermont) ^ King William 

(Virginia) I and Evans ton (Illinoi's). In such plans ^ the merit iticreaae 

becomes a part of the persbir's basic salary. ^ 

The major drawback to salary increases is that they are "permanent"; once 

giveh>^-it is usually quite difficult to withdraw them. If a- persoh^'^a high 
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performance in one year does not continae in fature years,- chat person wiii 
nevertheless continue to receive^ the increased salary. 



It is conceivable that the ent4,re salary could be related to perfbr- 

— - _•- ' ~ _" ^~ ~ ~ * ~ 

iiiance^^^Ebwever, in almost all actual cases^ the basiC; salary (even under pay-» 

_ _ -. - - ^_ _ . , ' ' _ 

f dr-perfbnnahce plans is still, determined to a considerable extent by Actors 

other than perfdnnahce^ such as longevity and education. Incentive plans 

subject only part of the^i^alary to' the perf brmance test • 

Seldom do plans provide for salary reductions for poor perf ormancei.^ 



There are exceptions however i King Williams' new* i>lan- could potentially 
involve sizeable reductions in salary if the teacher, has already bee}n awarded 
substantial ^^^ikry increases, bas^d-in part on perf bn&ance;. Thus, a teacher^ 
having a poor year could fall back to the basic salary scale, a possible Ipss 
of sevezial thousand dollars per year (see King Williafii; case report). The 
Bristol (Vermont) plan provides for salary changes of plus or^ minus 5SL(£RS 
1983^ p. 154). In the Parkway (Missouri) plan ^ a poorly performing teacher 
could regress up to one level, on the salary schedule^ but only once every two 

. . __ __ _ _ U^i^ 

years (ERS i979, p. 53). Note that :8tate law may ti^ibtt such salaty reduc* 

ttops. Thus for school discriccs in Missouri, under state law. a salary 

•* .- • 

decrease Is considered to be a demotion and must be accompanied by a po^ic 
hearing (a major undertaking). _ ^ ' 

The limited information we have indicates that even fvhere salary reduc*^ 
tidns are possible under the plan, ^t hey havjs seldom bej£n dmpl^ented. On the 
other hand I one district "^pointed out tha^ a person who consistently received 
ho perf brmahce'^related salai^ increase would, in effect, hbt be keeping pace 
with the cost of living and eventually would be released. Several instances^ 
where teachers * receive^^ho salary Increases at all were^ in fact^ repbrt^. 
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2» Bonuses > The majority^ of recciic taeric pay plans have used bonuses 
(over twd^fhiirds of the districts that we examined^ • These are bhe-tlme 
payments. Bonuses have the advantage that a person Is hot "permanehtly." 
rewarded for a high ievel of perfbrmahee that occurs in one year. The award 
is oniy for the particular year in <which oite person performs well. 

Bonus plans do no^ provide compensation reductions in periods of poor 
performance (other than not again receiving the boiius)^ Only part of the ^ 
employee's compensation is actually affected by the pay-for-perf ortnance* 
^ linkage^: when bonuses are used; the employee's basic salary is unaffected by. 

the pa^-fbr-perfbrmaoce 'process. 

. .\ . ■ ^ : 

3.' Av"gehu" bf: awards . Th^ "menu" apprbach is qudL:te unusual. The 
Cataliha Fbbthiir' school' district in Tucsbhy Arizona is the primary example 



that we found. (However , a second school* district also in Arlzona~fempe 
Union — has begun to use this approach; Bee ERS i983.) Under this type of 



plan, teachers^ave an opt^ion as to the form of the award they receiver In 

Catallna .the teacherp may choose to take their bonus in cash or- they may 

__ "_" " -■- - ' 

choose ^another form» l|icludlng attendance at prof essldnal conferences or funds 

to support classroom ptbjects, e.g. the purchase of iastructidiial/enrichmeiit 

_ _ _ _ __ _ _ 5^ _ __ 

materials for th^ &lassbbm (such as a micrb-cbmputer^ a telescope^ an 

_ . ^_ _^ ^ 

aquarium^ or a camera-*"'see the Cataliha case report^. i 

This approach has some Impbrtant advantages. It gives the teacher a 

degree of choice as tb the reward ahd^ in particul!ar ^. it inay appeal to the 

ihHereht professionalism b£~^4:eachers. ' Mahyv teachers have expressed concern 

' that they are prof essionals dedicated to teaching aikd that it is an insult to 

■ ■ . V ' ♦ . 

use money as an inducement^ Those who believe that money Is hot a motivator 
;for teachers are likely to find appealing the option of awards that afford 
opportunities for prbfesstonai developme nt or. classroom projects. These may 
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provide a more acceptj^ble reward to many teachers than dollars. In Cataliha, 
approximaite3^y 30% of the /awards for the^ first three years of the plan (through 
1982-83) were paid in cash (this percentage became somewhat higher in Jthe last 
year); about 50% were for participation in conferences or workshops; and "about 
13% were for classroom pro jects/materiaSs . Pjotential Arguments against such 
an approach are that all teachers should receive the same opportunities for 
prbfes&ibhal development and classroom enrichment materials > and that poorer 
teachers are those most in heed of prbfessibhal develbpmeht. 

In the private sectbr chere has been some experimehtatibh with bffering 
employees a menu of awards, but these have usually fb^cused on the choice of 
fringe benefits such as extra days off or extra pension payments. And various 
private sector compensation systems give managers' the option of deferred com- 
pensation. Such options have been tried on a few occasions in state and local 
government non-educational agencies. Only in the teaching area (Cataiina) 
have we found ait example of award options that can directly help the clients 
bf the service rather than solely benefitting the employee. This may be a 
unique opportunity for the teaching prbfessibh. 

It is dangerous to generalize from one case^ but the virtues^ arid appar- . 

ent absence of major drawbacks, make this an option that we believe deserves 

: _^ 

widespread consideration by school districts. 



We have not been able to find any significant body of research that 
indicates which of. these types of rewards is a better motivator ia or but of 
education. The menu approach is intriguing and warrants further trials and 
experimentation by school districts i Bona&es, with their feature of rewarding 
persons only for the year in which the meritorious performance occurs, ^eem to 
have ar. advantage over permanent salary inareases while being the preferred 



How Large Should the Awards Be? Should Every Recipient Receive the Satae Size 
Reward ? [ Z ' : 

There is cbrisiderable agreement in all sectors; private and public (both 

in educational and noneducational agencies), that awards need to be "large 

enough" to be adequate motivators But how large is "large enough" is not 

indicated by the research to date. The cost of the awards is the major cost 

factor for ^ school district using monetary incentive plans. Therefore along 

with the decision as to how many persons should get awards each year (an issue 

discussed in Chapter 4)^ the size of the awards is a matter of considerable 

importance. y 

Mb'rietary avzards of $200 to $300 per year or less are likely to have lit- 
tle mbtivatiohal value. incentives of about $1,000 and higher appear to avoid 
criticisms about size* We found awards of up to about $6,000, but in those 
programs (Houston and King William, Virginia) only a portion (about one-sixth 
and one-third, respectively) were directly related to perfbrmarice rather than 
factors such as years of experience and educatibri. Bryant Texas raised the 
size of its bonus from $600 in 1977-78 tb $1*000 fbr 1983-84. Houston raised 
its outstanding edticatibrial prbgress " bbnus frbm $800 to $1^000 for 1983-84. 

A schbbl district al^b needs tb make the related decision of whether to 
provide the same amount tb all teachers who merit an award or to give differ- 
ent size awards based on relative performance. About one third of the 50 or 
sb;examples reported by the Educationai Research Service in its 1983 survey 
(ERS 1983* pp. 22-28) gave a fixed amounti ^We found no research or other 
evidence pertaining tb tftje desirability of one approach ys, the others ^ 

Districts that used a more "finely tuned" performance assessment process, 
such as point rating systems, tended to provide differentiated awards i Some 
plans had fixed award amounts for each of several award criteria* thus cbm- 
binin^ the concepts of fixed and variable awards. For example^ Hbustori's plan 



(see case report) provides a fixed size bdhus to all teachers in a school that 
has made "outstanding educational progress'* and, in addition^ provides a 
variable bonus for attendance (the size of this bonus depends on the number of 
absences by the individual teache'r). In some plans, the size of the 
individual awards is riot determined uirtil the end d£ the school year, after 
the district knows how many teachers have qualified for the awkrd. 

Most districts budget a fixed maximum amount for awards* A few, however 
do hot* For instance^ Houston budgets an amount based on the administration's 

estimate of the number of persons lifcely to receive each size award* If its' 

; - 

estimate is too low, the district would need to provide the additional monies. 

More typically, however, djistricts fix the maximum amount of dollars that "can 

be awarded. An unusal case is that of Round Valley (California), which puts 

into its budget the maximum size award for each teacher in the schodjf district 

($2,000 each in 1983-84). 

What Prdpdrtidn of a Teacher's Cdmpensati^jn Should -Be Linked to Performance 
In essentially all the incentive plans we examined, drily a small prdpdr- 
tibn of a teacher's salary is actually determined by the plan. (Note that 
dismissal deci^^idns^ which of course involve a teacher's entire salary, are 
not a major element of incentive plans.) Even the largest awards are at most 
25 percent of salary, and award levels of 5 percent of salary or less are more 
common. As discussed earlier, no plans using bonuses and few of the salary 
increase plans affect the basic salary; they usually only add to it. 

The "largest potential salary swings we found were contained in the new 
merit salary plan for King Williara( Virginia) i i€~l:duid permit significant 
salary reductions (up to about $6,000) if a teacher does not^ maintain the 
performance levels that enabled the teacher to gain that additional salary. 
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Even in districts with comprehensive merit pay plansi teacher compensa- 
tion can be changed for factors other than performance, for instance longev- 
ity, education, and across-the-board (cost-of-living) increases . 

The continued protection of a large proportion of a teacher's 
compensation from performance cdriisideratidns is likely to dilute the motiva- 
tional effects of monetary incentive plans. However, there is It tie direct 
evidence oh this issue. Thus^ the inclusion of other factors in determiiuLrig 
cbmpensatibh* such as across-the-board increases taldhg with perf brmahce-based 
increments), is likely to reduce the opposition towards these plans by 
teachers' associations and by those teachers who do not feel that they would 
be likeiy to receive merit pay. 

Those districts that believe in the principle that teacher compensation 
should be based primarily on performance may want to apply more radical 

/ 

compensation prdcedutes such as larger merit pay amounts and inclusid.n of a 
prdvisidn for reducing pay if perf dnnaricis is ndt up td par in a year, but such 
steps will very likely also lead to substantial oppbsitibh. 

Should Merit Increases Be in Addition to Acrbss-the Bbard^ Cbst-bf -Living 

Raises ? 

/ _ - 

We are ndt aware of any plan that does not have prbvisibh for across-the- 
board increases in the ba^ic salary schedule to adjust for such factors as 
cost-of-living increases or to remain competitive in the market for 
teachers. A provision for such increases is likely to reduce the opposition 
by teachers and their associations. 

An important concern here is that tne salary levels for the school dis- 
trict remain competitive with those of nearby school districts, taking into 
account the merit pay plain TiacHing and"ailin±Tr±iHrra^-ve->-fieTSQnn — 
Ladue (Missduri) ^tiJ Summit (New Jersey) expressed concern about the presence 



^oxcLLaag_Shoulc^ the^a^rf^mano^ ^e^id^ Be? 

In all the cases that we examined, the awards are made at the etid of the 
scfebl year and cover the entire school year* We found rib iristarices where the 
performance periods were individual semesters* 

According to motivational thebry, the mbre quickly the reward follows the 

■ ' \. 

p'eriod of meritbribus performance, the greater? tfie motivational effect. The 

' y '-;/ . ' • : 

.longer the lag between the time of performance and the reward, the less the 

motivational effect. In government agencies other than education, shorter 

time periods — such as three or six months — have occssionaHy been usedi For 

example, a pilot incentive program for stati employment service offices in 

■ _ _ _f __ _ , _ • 

Kansas and New Jersey used a six**mdnth perf <5tlnance period, with bonuses coining 

shortly after the end of those six-month periods. 

For elemeritary arid secdridary schools^ however^ the ftall school year seems 

to be the appropriate peribd. The first semester cbritairis such distractibris 

as the Thanksgiving, Christmas, and New Year's hbliday seasons, making the 

i J , ; 

first semester too short a period to reflect performance in an adequate 
manner. Another major disadvantage in using semesters is the added effort 
that would be needed to make performance evaluations each semesters Frequent 
interim reports and the extra paperwork they caused were bne of the problems 
identified for the now terminated Lebanon (Connecticut) merit pay piani 

A related question for school districts is the timing of the payment 
relative to the etld of the school year. Many school districts make their 
determinations as to who merits the awards arid how much they will receive 
before the erid of the school year — for iristarice, sddri after final Spring 
ratirigt by stapervisbts or Spririg studerit test scores are received. Ladue, 
Missouri for example, requires the determiriatidri arid ridtificatidri df salary 



changes by the third Monday in March. Thus^ the perfbrinahce period often ddes 
not actuaiiy encompass the entire school year. On the dwher harid^ state laws 
concerning deadlines for notifying teachers aboat their emplbymeht status and 
contracts for the next year often play a major role ±nr determining when per-' 
fortnance evaluations are conducted and when notification is given concerning 
next year's salary. 

As noted earlier, it is geiierally desirable to determine and present 
awards as soon as possible after the necessary irif drmatidti has been received^ 
In most cases this means at least by the end of June of early July (as 
•deposed, for example ^ to delaying the awards until after the start of the next 
school year)- This will provide timely reinforcement of desired performance 
and will give teachers an opportuniry to use the information in planning their 
activities for the next school year. 

V 



CHAPTER 4 : 
Who Should Be Eligible for Awards? 

/ 

This chapter discusses issues relating to what types of persons should be 
eligible and how many awards should be given. - 

Should Individual or Group Incentives (or a Cdmbin^tidn) Be Used?^ 

One issue for a school district is whether the incentive^ should focus on 
individual teachers or should b^ Joint awards to groups of teachers* Most of 
the plans we reviewed were incentives for individual teachers , but there are a 
small number b£ group plans or at least plans with a group incentive cbinpdnent. 
^ in an individual incentive 'plan ^ whether a teacher receives ah award is 

based solely on the individual's own performance— -though the award may be 
based on how that individual performs relative to others^ In a group plan, 
groups of teachers (possibly including support staff) are evaluated and 
rewarded as a unit. Each group is evaluated as a whoie, and all members share 
in the awards. In most cases , all members of the group receive the same ^ward 
(though this usTially is nbdified for members of the group who have served only 
part of tht perfdnnahce period). v ^ 

We found hd pure grdup incentive plans for elementary dr secondary 
teachers, but some plans use group awards in cdmbinatidn with individual In'* 
centives. Houston has several separate awards. It awards up td $500 per year 
to individual teachers based bh their bwn absenteeism record (see case re* 
pbrt)i However, Houston alsb gives $1^000 bbhuses to each teacher in a ^schddl 
whose improvement in student test scores cbllectively earns the schbbl ah 
"outstanding educati^ai progress" designation. Teachers receive the bonus 
regardless of the performance of their own classes bh thbse test scbres; how'^ 
ever, they must individually have met certain eligibility prerequisites such 
as having had a satisfactory teacher Evaluation (see case report). Similarly ^ 

o 
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Dallas' new plan focuses on the entire school fbr awards based on student test 
scores, but to qualify ftir the awards individual teachers must also meet cer- 
tain teacher absenteeism and student attendance criteria. As in Houston , 
Dallas teachers must also have received a satisfactory performance appraisal 
from their supervisor to be eligible for the group award (see ERS 1983, ppi 
128-131, and Dallas 1983)^ Seiling (Oklahoma) also uses a combination of 
group and individual incentives. Participating teachers receive a bonus if 
reading scores in their school reach, the targeted increase^ regardless of the 
scores in their own classes* In addition^ teachers whose own class scores 
reach the target receive an additional bonus fbr each of their classes that 
reach the target (see case report). Round Valley (California) also uses both 
grbu^^nd individual awards. Twenty-five percent of a teacher's bonus is 
based on the success of a cooperative project in which the teacher works with 
other teachers to achieve a coinmon objective (see case report). 

Even non-monetairy performance-by-^b jectives plans can involve group 
incentives. Brown Deer (Wisconsin) teachers occasionally include joint objec- 
tives in which the teachers Cddpei;'ate towards a given goal. 

Group incentives have the attractive characteristic that they tend to 
encourage cbbp^atidn among teachers. Depending on the group dynamics ^ how-/ 
ever, they can water down the iiK>tivatidiial power of the incentives. An 
individiial's own perfdnoance Is oiily part of the totals aiid the individual's 
role will to a certain extent be hidden^ or at least bbscured^ rathir than 
highlighted as with individual incentives. 

Grbup incentives can alsb lead tb sbme inequities. Fbr example^ in plans 
such as thbse of Hbustbn^ Dallas, br Seiling^ teachers who individually per- 
form very well may not receive an outstanding educational progress bonus 
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because their own schools did not 'aeet the targeted test scores; simiiariyi 
teachers who perform without distinction on the test score criteria may never- 

( _ . . _ . 

theiess receive bonuses as long as they pass the minimum absenteeism and 
performance appraisal requirements. ^Notei^bwever , that by including these 
latter requirements, these plans etisure that teachers who do ndt^ receive the 
miaimum acceptable perfornance rating or do not meet the^bsehteelsm require- 
ment will not receive a bonus.) 

Despite these inequities, proponents of group incentives argue that over 
the long twxiy peer pressure within the group encourages poor performers to 
improve their performance. To some extent, the motivational power of group 
incentives depends on the size of the group, with very large groups tending to 
dissipate the Sbtivatibnal impact of the group incentive to a greater extents 
When a district believes this danger eicists (especially in large districts or 
schools), subgroups could be used composed of teachers physically near each 
other or teaching the saw grade or same subject matter. 

Groups can consist of teachers in the same schooli saSe; grade level, same 
subject matter, same floor, etc* Of the four districts we examined that had a 
group incentive component, three used the school as the group. Only Round 
Valley i California (a very sinall district) did noti There the groups for 
teacher cdbperatibn prd3ect8 were defined as groups of grades 4-6^ 7-8 ^ 

and 9-12). 

Individual incentives tend to put control over whether the individual 
receives the award in the hands of that individual; thus, individualistic 
persons many prefer it. Group incentive plans have the advantage that they 
are less threatening to individuals since the importance of the Individual is 
diluted; not receiving an award is not as embarrassing since an individual is 
just one member of a group that did not receive the awards This advantage may 



e particularly important in those districts where the .law requires identif 1- -v 

:"..,v. ^ ^ . .! 

^ ^.cation of persons receiving awards* 

X _ We found rib studies of public sector employees focusing on the question 
'^^fioT }atiet^r group or individual incentives are preferable and under what condi- 

• ^ 

tibris* Evidence from private sector studies indicates that individual inceu- 

1- ■ • ■ _________ 

^;l';^ tives are more effective than group incentives and that smaii group plans are 

I : imbre effective than plans involving large groups* (For reviews of this past 



.^reseajLch, see Sreiner , 1981, pp. 19-25, and Katzell et al * > 1975, pp. 

- - ^ 

Jlt^3^1,J • Of course^ teaching differs in many ways from ptivate sector white 
coiia*! J^rofessions, potentially limiting the transferability of ^he private 

sjector. findings. / 

' ' V - _ 

Should Pajcticipatibn Be Voluntary? , 

A ^chdbl' district can inatce participatibri maridatbry or vbluritary. Vbluh- 
r.^ / tary plaiig appear to be assbciated with reduced bppositibh to Mrit pay. 

^^acherg who blj^ect in principle to a pay-f br-perf bmance system can opt out 
if the plan is voluntary. Hbre impbrtahtly ^ a vbluhtary f>laii prbvides a, face- 
saying element fbr teachers who feel' they would not receive awards and would 
be embarrassed by not receiving an awards (Note that face-saving is much less 
Important in group incentive plans where the embarrassment is already diluted^) 
For some voluntary plans (such as in King William, Virginia)^ where parti- 
cipants go through a special evaluation to be considered for awards, the volun- 
tary feature permits the te^scher to avoid what may be a painful additional 
evaluation process. In other cases (such as Hbustbri) the teachers, by vbliiri- 
teering, do riot subject themselves to any addltibrial assessment process (see case 
report). The criteria (e.g. test scbres) are assessed fbr each teacher whether 
or ribt the teacher volunteers fbr the incentive plan. In Hbustbh^ in fact^ the 
teacher does not have to formally request to be a participant: in the incentive 
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award program until after the test scores and information on their absenteeism 
arp known. Thus, the voluntary aspect of Hbustoh's plan 'permits face-saving. 

Penn Manor , Pennsylvania, originally used a hbmihat ion approach; teachers 
had to be fdj^ally ndminated to be considered for 'merit pay. Teachers coaid 
hdminate themselves ^ and if ndminated by others, they could decline to be con- 
sidered further. The hcniihatidh procedure^ however, developed inequities and 
was drbpped\fbr the second year of /the plaui Participation in any given year 

is still voluntary^ The superintendent in Lebanon, Connecticut felt that the 

0 ___ _ ■ _ _ __ _ _ _ •_ 

lack of a voluntary elenffint in Lebanon's monetary incentive system contributed 

■i 

Xo the widespread opposition to the plan and its ultimate demise.' He recom- 
mends that participation in merit pay plans be voluntary (see case repbrtji. 

Apprbxtmateiy 20 percent of 50 or so school districts whose pj^aiis are 
identified in the Educational Research Service's 1983 survey indicat^ct that 
their plans were voluntary (EKS 1983). - 

The Education Cdmmissidn df the States recomxnehds that plans be vbluhtary 
for current teachers but mandatdry fdr hew teachers (1983^ p»5). This 
apprdach has the advantage that it dbes hbt impbse a hew element on teachers 
already i^ the system but lets prbspective recruits know ahead bf time «fcat 
the requirements will be. . 

Shbulfl Persons Other Than Teachers Be Included in the Incentive Planl 

Some districts include employees other than teachers in the same monetary 
incentive plan. For eacaiple, the plans of Penn Manor (Pennsylvania) , Lebanon ' 
(Connecticut), and Tredyffrin (Pennsylvjania) included such personnel as 
counselors^ nurses^ and library staff — but with a different supervisdry rating 
form for each type of jobi (Note that although neither of the latter twd 
plans is currently in operatlDo:^ their inclusidn df such non-teaching schdbl 
employees does not appear td have been a reason for their termihatibh.) 



We found no evidence that indicates whether such increased coverage adds 
to or subtracts from the effectiveness of an incentive plani- Neve^theiessi 
the school district is faced with this choicer It can inciude support person- 
nel and administrative personnel, especially if the plan uses the school as 
the evaluation unit in a^ group incentive plan. One option is to have a 
completely separate incentive system for support and adtnirlistrative persons; 
this appears to be the more usual arrangement. - 

For nbn-mbnetary perf brraahce-by-^bb jectives plans (discussed in Chapter 
7),. inclusion bf a school's suppbrt persbhriel is ciuite natural since the same 
basic jDrbcedure can be used for assessing each person '*s performance and for 
tailoring the criteria to each person's own jobi in fa^rf setting i'Sdividu&l 
targets for support personnel may be easier than for teachers i - ' ; 

How Many Persons Should Be Able to Recei^ve- A^rds? _ Shonld Therp Be, gotng Form 
QfZgaoti? ,^ • 

This is a very important issue; one that- has caused serious -problems jfor 
many past merit pay prograffls in educational and ndn-educatidnal settings 
alike. . ^ 

School districts may watit to constrain the number arid percentage bf 
teachers, receivi rig merit awards fbr brie or both bf two reasbris:' budget 11ml t- 
atior^ i-Qr_the belief that only some t earhorc will he siifffiriently ipprirnrJniifi 
^tb warrarit awards. If tbb many persons receive avy&rds^ it Is a sign (some 

_ _ _ _ ___ ___ __ _: ■ f ^ 

believe) that the evaluation criteria are too easy to meet* The federal ^ 
governmerit and many state ^rid local governments have used specific quotas in 



their merit pay plans, for example, a masSmum of 5%, 16% or 2Q% of their 
employees can be awarded extra compensation in a given yeari 

Though we have not found any systematic assessment of the effects of 
quotas^ it seems clear from the axmiiabie evidence on both educational and 



hbri-educatibhal merit pay plo^ns in the public sector that stringent quotas . 

_ _. . . _ __• 

have an adverse and often a serious effect. Employees involved in a pilot •» 

bonus plan in the city of Milwaukee^ Wisconsin objected strenjubusly to a 10% 

restriction on the number o^ recipients (Hatry et al ., 1982)>, Teachers asked 

to provide feedback after the first years of Houston's teacher merit pay plan 

complained strenuously about the policy of limiting awards to 50% of the 

schools, thus preventing uther schools that'ftight also be tneritbrious from ' 

receiving butstahdirig educatibnal prbgress awards. These quotas were subse- • 

quently dropped (schbbls now coTnpet;e primarily against their owS previous 

accompli shtnents rather than against each bther~see the Houston case report). 

However, one tight quota system that is reported to be popular ^th the 

f ._' ; _ _ 

Staff is tJhiit of Ambers t~Pelhatn, Massachusetts .with a maximum of 12 awards for 
^ approximateiy 275 eligible teadhers "(less than 5^) (ERS 1983, pp. 40 and 86). 
The key here may be that teachers are nominated either by staff, parents, or 
students, and the assessments are not part- of a formal teacher evaluation 
process that covers all st^ff. ' ^ " 

Bryan, Texas initially had a limit bf 20 percent oti the propbrtibn that 



could receive awards but itl 1983 increased it tb 35 percent. Bryan has raised 
its quota three tijnes since its plan begatii from 20 percent In 1977-78 to 35 

percent in 1983-1984. (It alsb has raised the size of its bonuses twice over 

^ 3 J, " , . - ^• 

that time* froth S600 tb $1,000; see case report.) ' - - 

■ _ ^- -_ * - - ^ 'i - - ' - 

The mafjbr prbblem with qubtras is that teachers wKb perform meritoriously 

can be excluded frbm receiving awards in a given year, diluting the emphasis 

bn merits thus depressing morale^ and encouraging backbiting and hostile 

attitudes towards those persons oy^^o^^s who received the limited number of 

awardsi The result caji be widespread dissatisfaction and dissention.. In * 

Arlington, Virginia, quotas on the oercentage of teachers rated "outstanding" 
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in a given year cmiSed considerable bbjectlbh frbm teachers (even without 
being linked .tew merit pay). (Arlington teachers were especially concerned 

because the ratings were based on qualitative judgments "tainted" by percep- 

l __ _ __ J ' 

tibns of favoritism^ etc. — Arlington^Educationai Association, 1983.) 

One advantage of quota^^is that they insure that the^ process will hot be 

J*. 

so corruptisd that too many people receive awards. For example , in federal ^ 

states and local governnients, there has often been such a large percehfage of 

people receiving high ratings that officials are concerned that if there were 

> .- • . 

no quota, many. people would deceive awards whb db hot ^really merit them^ Both 

Summit (New-Jersey) ^arid Tredyffrih (Pennsylvania) reported that one of the 

majbr reasons they dropped their merit pay piins was that : ultimately almost 

all teachers received about the same rating and almost everyone received a 

mei'it pa3nneht — thereby distorting the concept of the program as a merit pay 

plan (see case reports)^ thus^ vd.thout a quota system, districts will heed tb 

use performance evaluation procedures that minimize the likelihood that uhde- 

_^ • 

serving persons receive awards i 

The second reason for quota systems, especially thbse thSt ihvblve very 

- tight quotas— (f^^r inistance^those-^^^ of 5^10% of the staff 

tOvteceive awards), is that merit awards should" be provided only to really 

outstanding teachers. There is prbbably only a small percentage of butatand- 

ihg teachers in the strict sense. But there Is a danger with a tight quota 

t-Jiat the -same (butstandlng) teachers will receive the awards every year, 

f urther "aiminishing the incentive effect? on others. And for plans using 

supervisory and peer ratings, there e^sts the danger of "halo" effects; 

teachers >rated outstanding in the past will tend to get high ratings in future 

years, even in years j^hen they may not be performing in an 'outstanding manner; 

Gn the other hand^ if a district wants to use* the merit system to eticblirage 
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ail or most teachers to inprove tiieir performance; it is iikeiy to be ; 

1....: - ----- \ . - - ' > 

necessary to open xrp the quota to a substantially larger proportidri of the 

teachers so that more than jast those ^who are perennially outstanding can 
receive awards. c ' 

Quotas are also used as a budget/expenditure control device. School 
districts face a tradeoff between 'the number of bonuses and their size (see 
Chapter 3). The dilemma is that enlarging the pbtential pool of awardees, 
while at the same time keeping the size of individual awards reasonably large, 
increases the funding needea for the plan. As will be discussed farther in : 
Chapter 6, the district neeas to commit adequate funds for awards or the plan 
will be undermined i 

Quotas tend to introduce competition among teachers or, if group bonuses, 
among schools, especially if the quotas are small. In the private sectoF^ 
such cdmpetitidh is^ often Iddked oh as being constructive and appropriate. 
Th^ teaching prdfessibh is less used to such cdmpetitidh. Mahy^ if not a ^ 
majority i of teachers may dislike direct competition and even be upset by it, 
with cbunter-prdductive results. Other forms of competition witJi fewer inter- 
personal aspects may be more appropriate for teachers. Examples incite hav-- 
ing teachers compete' against a pre-set target (such as a minimum test score or 
other objective they have set' for themselver at the beginning of the year) or 
against their own historical levels of achievement (as in the Houston and 
Dallas plans )i - ' 

hbre direct competition, however^ might become acceptable to teachers if 
the performance criteria are perceived as being fair, if the awards are made 
more professional (such as with the Catalina, Arizona "menu" approach), and if 
enough awards are made available so that they woula not be limited to only a 
small, select group. 




CHAPTER 5 ' 
How Should Teacher Performance Be Evaluated? 
Evaluating teacher performance, whether for a merit pay plan or a non- 
monetary perfbrmance-by-objectives plan, lis prdbaSly ^e key stumbling block 
to incentive programs- All* incentive plans, one way or another^ have to 

assess teacher performance. To have a truly effective incentive plan it is 

. _ ^ _ _ - _ J- - ^ _ 3 

necessary that this assessment (a) be reliable and valid (i.e. evaluate well 

those elements that are supposed to be assessed), (b) be . reasonably compre- 
hensive (i.e. cover major aspects of^ a teacher's performance, even if not aii 
aspects can be measured)^ and (c) be^ perceived by the various participants — 
teachers i adiiiihistrators , the school board; and the community — as fair and 
reasonable. This is a very difficult and complex task. Probably the single 
most crucial decision a school district must make regarding incentive plans Is 
how to evaluate teacher performance. 

School districts have been wrestling for decades with the problem of 

m 

teacher evaluation. Tiiere is considerable disagreement and cbhtrbversy over 
various teacher evaluation methods. The problem of teacher evaluation becomes^ 
greatly magnified when the results 6f those evaluations are hot bhly to be 
uised for identifying prbblem teachers {as candidates for dismissal) and for 
helping to improve teacher perfbnnahce^ but alsb as a way to differentiate 
teachers as to cbmpehsatibh. It is clear frbm bur bwn past wbrk with non- 
educatibhal state and Ibcal gbverhmeht agencies^ as well as frbm the edu- 
cational literature and our examinations of teacher merit pay plans, that 
using" teacher evaluatibns fbr the purpose of determining compensation greatly 
magnifies participants' concerns over the evaluation process. 

The importance and complexity of this topic are indicated by the length 
of this chapter. in this chapter we discuss the approaches to teacher 
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performance evaluatib'^* and how te^acher performance evaluation findings can 
then be used to determine the size of a merit award. 

The selection by a school district of which particular procedures should 
be used to assess teacher performance is likely to.be the most important deci«- 
sioti, certainly the most technical, decision, that the district makes i one that 
can have considerable positive, or negative, implications- 

What Elements Should Be Lvaluated? 

It seejns universally accepted that the student is the= primary client of 
teaching and that ideally student learning is the central arid ultimate bbjec- 
tive of teaching, ana thus learning should be measured. However i agreement 
and consensus stops once one proceeds from this ideal. A key problem is the 
presence of many external variables outside the teacher's control th^t can 
affect student learnings At the extreme, if one believes that teachers' 
Ability to alter the amount of learning of students is negligible, then any 
justification for additional, perf ormance-based ^ compensation becomes highly 
dubious, especially from the viewpoint of the community. An underlying 
assumptidri in teacher incentive systems is that teachers can indeed through 
their activities have a ridri-trivial effect on the nature, quality, and/or 
quantity of learriirig by students. As we noted in Chapter 1, this issue has j 
been debated over the decades; it is well beyond the scope of this report to 
^et into this debate. Rather^ we accept the assumptibri- that teachers can have 
a significant impact on their students. 

But learning, i.e. the cognitive side of schbdlirigi is ribt the btlly out- 
come for pupils in elementary and secondary education. Thbugh m^ch less ofteri 
discussed, probably because of its vaguei^ss^ "affective" ^elemerits are alsb 
generally of concern i These are usually defined to include the attitudes and 
values of pupils, including self-esteem, self-reliance ^ ability tb relate well 
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to other students and adults (i*e; interpersonal relations), "work" habits, 
and such values as respect for home, law, and cbtantry. (See^ for example ^ 
Jdhrisbh 1974). Different persons will give different weight to the schddl^s 
role in imprdvirig affective dutcdmes--v7i th some perhaps beliievihg that affec-. 
tive dutcdmes are the responsibility of the home and not the, school. We fouitd^'*^ 
little explicit attention to them in oar examination pf teacher incentive 
programs, and even in most of the teacher evaluation literature we examined. 

There is a third major class of elements on which to evaluate teacher 
performance: "process" indicators. Many believe that student learning can 
not be validly measured, such as 6y test scores, or at least that test scores 
haVe substantial limitations for teacher incentive pay purposes, thereby re- 
quiring other, or supplemental, perf ormahce indicators. Aiid others believe 
that in any case a teacher's task "is dhly td provide the best possible 
envirdnineht ^ hdt td guarantee that the results will be effective . . ." 
(Scriven in Millman^ 1981. p.25p). Frdm this it fdlldws that perfdrmahce 
assessments need to assess the prdcess by which teachers wdrk with their 
students. This leads to performance evaluation procedures that measure the 
teaching process. , 

Teacher compensation is also often linked to exch factors as advanced 
educational credits, having passed a teacher proficiency test, participation 
in extracurricular activities such as coaching and bandleading, taking part in 
professional work outside of regular school activities, teaching special 
subject matter where teachers are in scarce supply, or teaching in locations 
which otherwise would not attract quality teachers. Though there may be 
reasonable justification for providing additional pay for such activities, 
these generally are hot considered td be performance characteristics, and we 
dd hdt cdnsider them such in this repdrt. Hdwever^ as is hdted in a number of 



the case reports^ many school discriccs have included some of these criteria 
in their incentive plans, either as an additional basis for extra compensation 
or as prerequisites (eiigibiiity criteria ) for individual teachers before they 
can receive awards for performance* Extra payments for extra ^responsibilities 
are a characteristic of naster teacher and similar career ladder plans. This 
type of plaQ^is quite different from the incentive plans that are the main 
fdctis of this report, as discussed ill Chapter 9. 

Underlying the debate over whether dutcdmes (whether cognitive dr affec- 
tive) dr prdcesses^ or both, shduld..be evaluated is the questldh df whether 
teacher awards should be based only on criteria that the teacher completely 
controls. In the private, business, sector many compensation plans, partic- 
ularly those for managers, are related to business dutcdmes that are affected 
by many factors not completely under the managers' control (such as national 
economic conditions); The underlying concept is that when outcomes are good^ 
personnel should get higher rewards than when outcomes are bad, even though 
external factors may have played a major role in determining the outcomes. 
Windfalls (and their reverse) are accepted as part of the game; (The reasons 
behind the outcomes can be considered when fine-tuning the amount of the 
awards.) School district officials aiici teachers using merit pay plains, 
particularly those whose performance is at least iti part based on dutcdmes , 
shduld recognize that such windfalls will almost certainly have td play a part 
« in determining rewards. (Districts^ however, shduld to the extent possible 
explicitly consider such "external" factdis as student characteristics when 
evaluating teacher perfdrmahce^ as will be discussed later*) 

tfhat type of teacher Performance Evaluation Procedures Should Be Used? 

The fdlioWihg^ are the primary teacher perfdrmance evaluatidh prdcedures 
that are candidates for teacher incentive pi^ns: 
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li Ratings by superNasbrs/adininistratidn sltaff 

2m Ka tings by peers (including outside experts) 

3. Ratings by students 

4. • Ratings by parents 

5*» Student achievement test score results 

6» School district record data 

7. Comparison of actual achievement to pre-selected targets 

^ ■ r ■ 

Exhibit 2 lists these, evaluation procedures and indicates the type of 
element that, each measures. A school district can use one or some combination 
of these procedures. The following sections discuss each of theses but first 
some overall comments. 

. I 

The choice of procedure by a school district should depend on the follow- 
ing Characteristics of these procedures: 

b What the procedure measures. A district wanting ^tc focus on student 
I'^rhihg outcomes would focus on such procedures as examination of 
student test scores rather than supervisory or « peer ratings; a 
district believing that the ihstructibhal prbcess is the important 
issue would do the reverse.^ 

o Reliability/validity of the procedures. 

6 Acceptability, by participants ^ i.e. teachers and administrators. 

b tost of the procedures, 
in the following sections we ast tempt to address these characteristics for 
each procedure i By and larger however, the amount of substantive evidence and 
empirical research on these procedures is. quite limited, particularly in their 
use in teacher incentive pi? i There has been considerable research on the 
relation of various teacher instruction practices to student learhingi A 



number bf schbbl districts introducing supervisory or peer rating jsystems, 
especially, thbse using structured classrbbm observations, have drawn on this 
wbrk in develbpiSg their instruments. Our reading of this research , however , 
indicates that there are few definitive findings about the validity and relia- 
bility of these approaches br the instruments that ^ have been used to make the 

^ , ' _ _ _ ^ 

assessments. (See^^fbr example^ Anderson 1982; Darlirig^Hammorid, et al. 
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Exhibit 2 

E valu a tion Procea ures arid What They Measure 



Frdcedure 



What Kinds of Elements Does It Measure 



1. Supervisory Ratings 

2. Peer Ratings 

3. Student Ratings 

4. Parent Ratings 

5. Student Achievement fest Results 

6. School District Record Data, 
e«g» attendance of teacher, 
attendance of students, accept- 
ance at colleges, etc. 

7; Comparison of Actual Achiev(sment 
to Pre-Selected Targets 



Process 
Process 

Process and/or Outcomes^ 
Outcomes and Some Process 
Outcomes (Cognitive Only) 
Process and/or Outcomes^ 



Pr-ocess arid/or Outcomes^, 



1. Whether process or outcome elements^ or both^ are measured depends on 
the specific questions asked of xhe students (#3), or on the specif ic. objec- 
tives selected for the teachers:^I#-7) i 

2. it is not clear whether an element such as student attendance is a 
process or outcome characteristic. 



1983; :eentra and Potter i986; ERS i978; Levis i982; Mtllxnan i98i; Peterson and 
Kaachak 1982; and Walberg 1974*) Research on these procedures when ased ±n 
the context of teacher incentive plans — whether monetary or non-monetary 
plans — is almost nil. Some work has been dVne on testing the reliability of 
particular data cbllectibh instruments (such as for structured classroom ^ 
observations and student rating questionnaires^ i but much less testing (if 
any) has been donjon testing the full set of procedures (of which the iastru-^ 
ment is only one part) in settings involving teacher merit pay* For example^ 
would peers alter their ratings if they know that those ratings are going to 
be used to help establish compensation for their fellow teachers? Will stu- 
dents or teachers behave differently during the testing process if test scores 
are used as a major determinant of teacher tdmpensation? 

Despite the many trials of teacher incentive systems that have occurred 
over the past decade (with a few even older) ^ ve found in bur review no exam- 
ples of multi^ite comparative studies systematically collecting data on the 
impacts^ validity/reliability ^ br costs of such plaiis. Some districts have 
attempted their own evaluations. These .have generally lacked rigor^ and 
usually have lacked adequate baseline data from which changes due to the new 
program ml^t have been /inferred* In the: few instances of local district 
evaloat ions that we foand, there has been no explicit^ systematic att^pt tb 
identify other piausibie explanations^ i.e. confounding effects that could 
explain observed outcomes^ 

thus the currently available^ empirical Information^ information based on 
actual Jield experiences— information critical to helping school districts 
learn from these experiences — ^is highly deficients The' key evaluation crtte- 

; ^ _ _■_ . 

ria these days appear to ,l>e whether a particular plan has survived and for how 
long and how articulate the proponents (or opponents) of the plans are* iii 



:? ; 35 • ^ . , . 

bur brief examiriatibris bf a number of district efforts; we sought evidence as 
to the impacts of the plah^ such as bn student learning, teacher reteritibh 
rates, ability to hire quality teachers ^ teacher and student absenteeism, and 
changes in teacher practices that might have been associated with the pl^n» 
The information we found to be relevant is included in the following sections 
and in the case reports in Part ii# 

Superv i sory Rating Procedure s > - 

Here we consider procedures that involve a formal rating of individual 
teachers by supervisors, such as a principal, department head, assistant 
principal, assistant superintendent, or other administrative personnels 

Rating by supervisbrs is the traditional form of teacher evaluations 
"The principal writes a shbrt statement regarding the teacher's instructional 
competence arid ability to wbrk effectively with the staff. This evaluation 
statement is based on all interactions with the teacher during the year, 
including classroom bbservatidris" (Niedermeyer and Klein 1972, p. 102). 

The Educational Research Service reports that sbme form bf stipervisbry 
rating was reported by almost all of the over 1^000 school districts respbrid-* 
ing to its mail survey of June 1^17 (ERS 1978). ; 

In recent years there have been attempts ^at more systematic classroom 
observations, using structured instruments describing specific behaviors arid 
perhaps more systematic sampling procedures. Examples of stach iristrumerits are 
given- in ETti 1978, EifeS 1979, EHS 1983, and Coalition 1983. These have been 
used in such districts as Bryan (Texas), Catalina (Arizona), Lebanon (Connect- 
icut), and Tredyffrin (Pennsylvania) — see case reports, in some of these 
cases an explicit attempt was made to develop the instrument based on past 
research as to which classroom characteristics were associated with improved 
student learning (e.g^. Catalina). A major example of this approach is the new 
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performance evaluation system being developed by the Cbalitibh for the Devel* 

opment of the Florida Performance Evalaation System (1983). 

Principal advantages of having supervisory/administrative personnel 

evaluate teachers are the following (Darling -Hammond, ^et^ ^al^ « 1983; tewis 1982; 

arid Peterson and Kauchak 1982; these same references are also sources for the 

disadvantaged noted below): 

d These persons have regular contact with the teachers being evaluated 
and^ to the extent classroom dbservatibris are used, can see the ^ ^ 
teach.er in action; these persons are familar with the school and 
cbminunity gbals and values. 

- o By and large this is the procedure that has been used by.'^lmost every 
school district for a long period of time §nd thus in itself ira^y do 
- little ^o upset the apple cart. ' ' 1 

o it provides an opportunity for the evaluator to prbvide cbnstructive 
feedback to the teacher to improve the teacher's fJerf oinnahce. 

o These personnel as part of their regular jobs prbbably need tb evalu- 
ate teachers periodically anyways ' 

There are, however, some substantial problems with this approach that 

have been widely noted: 

b There is considerable question as to the validity of the components 
of the rating instruments as to their ^relation to student learning. 



b There is^cbrisiderable question as to the reliability of these proce- 
dures. In the ihstlruments that -we have seen the rater still has to 
make highly subjective judgments regarding the particular elements 
being rated. - 

b Classrbbm bbservatibns of teachers tetid to be infrequent , perhaps 2-5 
times a schbbl year and then for hbt mbre than one hour at a titae. 

b Most >bf ten (but hot always) classrbbm bbservatibhs are announced 
ahead of time. "In either case, the presence bf an observer can 
affect the behavior of both the teacher and the class'^-thus throwing 
into question the representativeness of the bbservatibhs 

o To increase the likelihood of interrater reliability bn any giyeh_ 
instrument, special training is likely to be heeded peribdically for 
those doing the ratings (Evertson and Holley in Hillman, 1981). 

o Where only one person, such as a teacher's supervisbr i dbes the ^ 
fating, there can be problems v^th perceptions by teachers of favbr^^ 
itism or general unfairness, and of lack of skills in doing the 

. ... ^ W 
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evaluations; (One- school prbfessibhaiy hbted^ to us that principals 
tended to be highly stressed ^ under-trained, and have a tendency 
towards favoritism.) 



b The evaluations can take a considerable amount of saperyisory/adiaini- 
strative time. * 



EKLC 



Procedures that use some form of systemajtic rating/ot^servatidn prbcedure 
alleviate (but do itot eliminate) some of the/above prdblems. Hdweverj as 

. K : • _ / , ■ ; 

noted above, the rating instruments still af/pear to leave much tb be de^ ' 
sired. Fbr example, an instrument may ask the rater tb rate the teacher's 
"s^ill in motivating papii interest," but no specific guidance is^prbvided as 
to how to assess this nor what specific behavibr warrants what specific grade 
on the rating scale. This lack of specific "ahchbi/ihg\of the response 

categories on the rating scales is typical even with newly developed instiru- 

_ _ _ \ . _ _ _ _ • 

ments and has considerable pbtehtial for leading to different ratings by dif- 
ferent persbhs rating the same behavibr--apd 'at different points in timei 
This gap alsb means that teachers do hot fcibw v^atj specific practices/beha- 
vibrs are needed to obtain high ratings. Sor have ^bst of these instruments 
been tested as tb their relatibh tb student achieyemeijt. Where there is more 
than one persorn doing the ratings for any one teach^rT^ or for a group of , 
teachers whb are "tbmpeting" for the same merit dollars, the lack of specifi- 
city of rating scales becomes an even greater problem, creating inter-rater 
reliabtlity problems^ The Florida work mentioned above for the new perfbr- 
mance evaluation system is testing interrater reliability, but thus far this 
has been done primarily in "laboratory" settings using videb-tape sessibhs 
father than actcrai in-the-field observations. 

- Problems of poin\s of view bf particular evaluatbrs can be alleviated by 
having more than one persbn rate each teacher on the assumptibh that bveralL^ 
the average rating will tend tb be mbre accurate. We have hot found completed 
research that shed much light bh the imprbvemeht ^ if ahy^ bf evaluations fbr 
inceiuive plans based bri use of miiitipie raters. * • 
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There* has been debate as Cd whether certain types of characteristics 

/" ' • . ^ ^ .- 

shdula be included bii these instrutnents » inciuddtlg personality factors i 
physical appearance factors , extracurricular wbrkj etc. -'Ladue teachers^ 
responding to ^ 1983 bpiriibh survey bh its plan* indicated that characterise 
tics relating to effective classroom teaching should receive major considera- 
tion, over such other factors (Ladue 1983, p. 29y* badue administrators , 

v 



agreed that classroom teaching should receive the %ost emphasis. 

Scriven stated that "using classroom visits by ;tpiieagues or administra- 
tors are a disgrace. 'V The visit itself alters the teaching, especially when 
pre^^announced ; the number of viSits is too small to be accurate for generali- 
zatibns; there may be persbrial biases arid prejudices, and "nbthiii^ that^cbuld 

' .... M.-; 

be observed in a classrbbm c^n be used as a basis • . • tb any cbhclusioh 
abbut the merit bf the teacl^ihg" (Scriven 1981^' p. 251). 



A majbr cbricerh bf the National Education Association is thatV^ahkihgs S.-^ 

/. ' ^ : ' -^"".^ ' 

fbr merit pay wbuld become /a vehicle f or favbritisxn debilitatirig*;tb teacher 

-------- - / --- ^ 

morale (The Washingtbri Pbs/t^ December 7^ 1983^ pyA-6*). *- 

~ i - 

Pbpham pointed out in 1974 tha^t use of dbservatibh 63^6tiemS| by focusing 
bh process and riot producj: criteria^ mearis- tha%vybu are assuming th5|^' the * 



process criteria bring a)>out desirable products (Pbpham :i974, p»143)» He« went ' 

^5 / ' - 



on to sayHthat because ob!servationai evaluation criteria usually need to be 

: ■ -I - 

spelled out in advance, this invites teachers to alter Their observed 



classroom behavior i '.^^ a^- 

I f \n • ■ *' 

Most school districts that we examinejl used .classrodm observations that 

/ — • »^ -! ' ' 

_ _ - ' - - - - ■- ' - - * . - - - 

were annouuced ahead of /time. An exception is KJ^tg William (Virginia) which 

has one announced and oiie unannounced observatl^ by the superintendent - 

central bffice staff (see case "repbrt ) • « ^^J^T* . 

1 



Teachers from whom we received informal information point out that some 
teachers e\jen though displaying what on the surface may appear to be uridesir- 
able characteristics (e.g. they may be disheveled i or notice articulate in the 
cultured sense) may actually perform very well arid ericburage student learning- 
"at_^least with certairi studeritSj such as certain cfisadvantaged students* 
Teachers who may appear lazy in their instructional methods (such as by makingii 
relatively substantial use of films) may actually be quite effective in 
helping students, the point' being made is that different types of teachers 

may be better with different types of students, and it is dangerous to become 

i ' 

too dogmatic about certain classroom characteristics, especially those involv- 
ing physical appearances and personality. Several reviews of studies have 
reported that teacher classroom behavior is a p<yQ^ predictor of^ student learn- 
ing (Miliraan 1981, p. 156). - ^ 

barling-Hammond, et^ ^ » (1983) in their review of research firidirigs ;bri 
the relation of teacher characteristics arid teadher behavior versus studerit 
outcomes and student learriirig Iridicate gerierally iriQorisisterit arid often con- 
tradictory firidings iri the: "process-^product" research. They found that even 
broader patterns of teacheX practices ap|)ear riot to be uriiformly applicable to 

dif fererit^rade levels^ subject areas^ arid teacher situations or types of stu- 

__ . • _ ' _ _ X 

derits. (For example^ such characteristics as teacher "indirectness^' that use^s 

more bperi-erided teaching to explore ideas may be effective teacher behavior 

for sbme 'types of students with certairi SES arid psychological characteri^ics, 

but the effectiveness might vary for students of different grade ^levels and ^ 

subject 'areas). They indicate that generalized rules for teacher behavior may 

o _ - _ ^_ - _ _ _ ^ __ . ' . 

ribt be r^lialSle in cbinplex and variable education environment Si 

Given that ratings are to be done, however , teachers app/ear to prefer a 

■ " ■ - ■ } ' 

rating from a supervisor or from the administration, such as their principals 



The American School Board Journal survey of teachers in May 1983 found that 

39 percent wanted t:l)eir% principal to be the evaluator, 25 percent preferred 

« 

their peers, with 27 percent preferring other admihistrative personnel such as 
department headi or- curriculum specialists (Rist 1983). This preference for 
principals was expressed more by teachers of lower grades, possibly because 

the elementary schools tend to be smaller with more personal contacts with the 

— - - - - ^ 

principal or because of differences in organization. 

--_ _ _ ' _ __ _ _ f _ 

Glass believes that past failures in such ratings are primarily because 

_ > _ _ 

of vague i general definitions of behaviors that are to be rated and the lack 

of rater training (Glass 1974); He believes that ^these can be rectified. 

_ i _ : _ • ' • — ■ _ ■ . * - - * ■ : 

(However^ he prefers an outside observer to somebody from the school 

administration^ and he made these points in the early 1970' s. It does riot 

\ • 

appear from our observation that the problems he noted then have been 

corrected to any sigrrf:f,icant extents) - ' - 

Rating teachers is likely to be one. of the hardest things principals and 

administrators doi. It is also one of the most unpieasan^ chores, especially 

for rating ••prdblem" teachers. Few administrators have had adequate training , 

in handling critical employee appraisals Such reviews if not handled 

properly can be counterproductiye. " ... 

Perhaps as a result of this, raters often tend to rate personnel highly,. 

thereby avbidirig debate with, .and hostility from, the peVsdtiS being appraised. 

Because Jsupetvisbry ratings are geherally -based heavily on judgment J ev^en if 

systematic classroom observation procedures are used^ the ^ratings caH be dif- * 

f icult- to support unless the supervisor has carefully documented t^aeher prob- 

lems throughout the year^ a task which itself can be quite time-coneuiningi 

Some of these rating ^teps are probably necessary in a, school district 

even if pay is not being linked to the ratings, but the link" to 'pay e^aijep^ 

. : . * • 

baties this problem considerably i 



^1 



Mariy^ if hot most supervisors ^ we suspect, would much prefer for monetary 
incentive plans evaluation procedures that provide more ^s^bj^ctive , supportable 
information that reduces the need for adversarial debates (Barrihgtbhi * ' 
Illinois School District, 1969). ^ 

What does ail this mean? The use of supervisory ratings is the most pop- 
ular form of teacher evaluation. Our limited review indicates it is also the 
most popular form of Assessment for performance evaluations used in incentive 
plans. But there are also indications of major problems among teachers with 
such ratings for monetary incentive plans. Criticisms 6f the subjectivity of 
such ratings were major reasons for dissatisfaction among tieachers in both 
Lebanon (Connecticut) and Tredyffrin (Pennsylvania)^ — both of which plans were 
dropped. These problems may be particulariy present when other problems exist 
such as l^ck of substantial teacher participation in developing and implement- 
ing the plan. The validity and reliability of the supervisory ratings 
approach is ±xt considerable questions 

^ The fact, however, that supervisory ratings ^re cdmmbh for teacher evalu- 
ation purposes indicates that the inajority of school districts probably are" 
iikeiy to Jbe predisposed to using this procedure. Such school districts 
should seek instruments and dbservatidh procedures that have at least some 
research basis for their components. The district will need to adequately 
train the raters in the procedures so that different raters (e.g. different 
principals) within, and across ^ schools in the district will tend to give 
similar ratings for similar performance. We believe that researchers and 
school districts need to put more effort into developing specific anchored 
definitions for each characteristic being rated and for each grade on the ' 
rating scale ^ both to improve the reliability of the ratings and to let 
teachers know more specifically than currently is the case what specific 
performance is necessary to achieve high ratings^ > _ 
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Ratings By Peers , ; 

Peer ratings are used here to refer to ratings ^^f one teacher by other 

_____ _ . . > . 

teachers or by outsiders, even those outside the school district^ that ^are hot , 



isupervisors or administrators in the school district. 



There appears to have been little research on peer ratings and a paucity 
of infonnatidn on the state of peer evaluation (see, for example, McGreal, 
19b3i p. 126). The chapter on peer reyiew in the 1981 Millman Handbook is on 
college teaching, not elementary and secondary (see French-Lasdvik in Millm'an, 

1981). 

The primary advantages of peer evalutions are: 

Peers, especially if from inside the school and district^ are 
presumably familiar with the teacher ah^ the locality. 

Peers are likely to encourage prof essibhalisin in the ratings (and 
thus^may also be less threatening). 

if' peers are from the teacher's subject area^ they may be able to 
give highly specific suggestions for improvement. 

Using peer rating as part of monetairy incentive pians^ however, can also 

present major problems and. concerns among teachers, especially between^hose 

rated and those doing the ratings. (For example, in Ladue, the high School 

faculty has been consistently opposed to peer evaluation.) Problems in' i^ide 

the fdllbwing; 

b Teachers may see subh ratings;^ as a popularity contest based on • 
friendshijp or general popularity or other irrevelant factors, thus 
causing morale problems and mistrust among cd-wdrkers. 

o Reliability problems may be particularly bad, unless anchdred 

instruments and substantial training is provided to raters. Peer 
rating systems will hbrmally involve multiple raters (ixidre than 
usually used for supervisory rating procedures). 

o The raters will need substantial trainings especially ''' special 
instruments are to be used. As indicated in the discuboibn bf 
supervisory ratings, to obtain adequate comparability among raters a 
structured instrument will likely be needed. 
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6 Th. validity and reliability problems discussed under supervisory 
ratingr also .apply to peer ratings. 

trained, making ratings, and probably discussing ratingb 
rated.' This could be a considerable time requirement. 

McGreal U9b3), in pointing out problems Buch as these, comes to the 
conclusion that peer ratings are undesirable .nd unrealistic for t.acher 
.valuation. Note that he is addressing the i..ue of evaluation only, not 
performance assessment for incentive pay purposes which is likely to 
considerably worsei> any tensions involved. j. 

Th. Educational Research Service in its June 1977 survey found that only 
33 of slightly over 1.000 reporting school systems (3 percent) reported the 
use of felloe teachers for the forH>al evaluation of' individual teacher 
performance (ERS 1978). We found it. use, and a partial one at that, in only 
one plan that ^e e^mxned (King William); it is discussed below. 

Darling-Hamn^ond ^ • in their review' of past research reported mixed 
reviews for peer ratings. They point out that peers are in a position to 
assess •omp^cenC, but use of peers is not generally recommended for personnel 
decisions (presumably including com^^ensation decisipns) becau.. .f "divergence 

of criteria" (1983i p.307)i 

Petefsoi and Kauchak (1*82) ^re ibre Saiiguiile ab5«t peer review foi 
teache. evai»ari<... b.lieving.that it briag. the expertise and experience ot 
t„. teaching profession Ih as does SS o.h.r .i.«s.ent technl,„i. Hb.e,er, In 
their o« revle» of Eesearch on It, they also reported It as "one of ths sore 
o-ndeveloped and andef-research.d areas of teacher evaluation." They aUb 
cpfiasUed th. problems of lac. of reliable p^ocedur... c,edtbi«.:y to oUt.lde 
aoaiences. «d the need for ..t.n.lve teacher preparation foF pier eValn.tlo„. 

. ' « ,<f npprQ in the same subject area giving 

But they feel that the advantages of peers m tne 

V i.si,fiai fhev believe that some of the 

highly specific feedback are substantial. They oeiie 
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methbdolbgical problems can be solved with increased attention to the stand- * 

ardization of these procedures and that its credibilitiy should be established 

by the use of other forms of corroborating data such as student reportsi They 

suggest that to implement peer review, it will be necessary for admihlstratbrs 

to review power relationships that currently exist in schools. We note that 

they are speaking of peer review in the context of annual teacher evaluation, 

not necessarily for the additional purpose of using those evaluations for 

compensation decisions. We suspect they, would be less sanguine if peer review 

JL5 Usee rbr this additional purpose. ' - 

A variation is to use combinations of teachers and supervisors /admini- 

strators. This is recommended in the Governor's "Pocket Guide" (October 

1983), which suggests mixed teams of evaiuators including teachers because 

"good teachers recognize good teachings" -it ai-so recommends that several 

visits be made by the evaiuators throughout the year. 

King William (Virginia) in its new program uses three evaiuators, two of 

- _ - - '' . . _ _ _ 

which are central office staff members. The third is sdmedne outside the 

school district chosen in part by. the teacher from a superiiitehdeat 's list of 

approved evaiuators. University professors have been chosen most frequently 

by teachers as their third evaliiatbr (King William case report). 

From a technical viewpoint ^ peer ratings have many of the characteristics 

discussed in the supervisory raLihgs. The problems of instrument validity^ 

inter-rater reliability (discussed above )i and the focus on process rather 

than product thereby assuming that the characteristics assessed are strongly 

and positively correlated with outcomes, are also inherent with the use of 

peer rafings. 

Those districts that choose to use p^ers for ratings will have to decide 
how many peers there will be^ how they will be chosen, from what pool of 
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persons, how frequently the reviews should be dbnii specif icalXj^ how they 
should be dorie^ especially to what extent they should be based on classroom 
observations versus by casual informal contacts througlidUt the year. We found 
little empirical /evidence on these questions. In addition^ there are few 
experiences as /iready noted and most of these are in systems using peers for 
teacher evaluations only, rather than for evaluations related to compensation, 

Scuaent ratings involve handing out questionnaires to each student in a 
class (near the end of the school' year), collecting thefi in a way to a?^ are 
anonymity , ""and tabulating the responses. 

The principal advantages of student ratings are: 

b They provide information from the perspective of the primary clients 
of teachers., a perspective not obtainable by other procedures. Their 
perspective is based on daily observations of the teacher for 
extended periods^ of time. 

b They can obtain information on a wide variety of teacher performance^ 
both 6utcome_and process-related aspects-. On outcomes, they can 
prbvlde information relating both to teacher helpfulness in learning 
^nd on affective aspects i infornstidn on Effective aspects cannot be 
obtained by most other teacher evaluation procedures. Process 
subjects that can be sought include isuch information as the degree of 
rapport, motivation and communication engendered in the rlassrdbm. 

^ froB students, particularly on classrbbm characteris- 

tics^ can give teachers specific inf brmatibh, helping them improve 
their own future performance. 

The primary disadvantages of student ratings are: 

o There are questions as to how meaningful such ratings are (but note 
that the research cited later indicates high reliability and 
cdrrelatibri with such procedures as supervisor ratings). 

d Teachers may feel uncomfortable with the concept of student ratings. 
Ladue (Missouri) teachers responding to a survey to evaluate its 
merit pay plan indicated oppbsitibh to any use of student ratings 
(Ladue case report?. (Participation by teachers in development of 
the questionnaire may be a way to reduce their apprehensions.) 

b Students may use the opportunity to reward or punish teachers they 

lifee (don't likei^ especially when thiy become aware that the results 
will be used to affect teacher pay^ 



There has been a considerable amount of research on student ratings but 
mostly for higher education. There has been a small amount of empirical work 
done oh student ratings for teacher. -evaluation in elementary and secdridary 
schools. 

Most of the research]; both on elementary and secondary and college 
levels, indicates that student reports can be reasonably reli'al^le (Aleamdril 
1981; eoher 1981; Dariing-Hamiaond 1983; McGreal 1983; and Peterson and Kauchak 
1982). - ' 

Peterson and Kauchak (1982) found in' their review of past research Jthat 
ratings from older pupils were quite reliable and that even teacher ratings by 
younger students, down to Grades 2-3, are valid. (They primarily reference 
the work of R. Haak^ D. Kleiber^ and R. Peck^ "Student Evaluation of Teacher 
Instrument 11^" Austin^ Texas: R&D Center for Teacher Education^ 1982.) Shaw 
(1973) also reports the use of student ratings for Grades 2*12 by Kalamazdb^ 
Michigan (Shaw, 1973). 

_■ _ ^_ [ .1 

The research findings indicate that students rate teachers fairly without 
being affected by background factors or by the grades given to them. Again, 
however, we note that these studies are not done in the context of the use of 
the infonsatlon for teacher compensation purposes. Aleamonl (1981) in his 
review of a number of college level uses of student ratings reported that most 
studies found little or no relationship between the ratings of the students 
and the grades and marks received by the students. Cohen (198r, p. 305) in 
his analysis of 41 studies, also at the college level , found that "students do 
a pretty good job of distinguishing among teachers on the basis of how much 
they have learned." In both Sah Mated and Lds Altds (Calif drhi'^) , students 
themselves analyzed relationships between the ratings and characteristics of; 
teachers and df students dding the ratings. They fduhd that the fdlldwihg 
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characteristics were not correlated: teacher sex, academic degrees, years of 
experience, or grade point average of the students doing the ratings (Shaw 
1973); 

School district use of student ratings to evaluate teachers is small. 
Ttie Educational Research Service's June 1977 survey found only 19 school 
systems of over 1^000 (2 percent) reporting their use in foraal teacher 
evaluations. We have found no district using student ratings as part of its 
incentive plan. It is not clear whether even those that report using student 
ratings for teacher evaluations actually use them on a regular basis and for 
all grades and ail classes. 

in an early effort, DePaal University developed an instrument for stu- 
dents in Grades 9-12 to rate teacher effectiveness. It made a number of tests 
of the validity of the instrument assessing correlations with teacher assess- 
ments, and principal ratings o.' teachers. Correlations of over .80 were found 
for the 40d teachers covered in this study (Shaw 1973 and Blanchard 1967). 

The American Association of School Administrators' 1981 survey fdutld that 
where studient evaluations are madej they usually are only for the use of the 
teacher. Only rarely do student evaluations become part of the evaluation 
record (Lewis 1981 ^ p. 35). Clearly the use of survey instruments with* 
feedback provided only to the teacher will be much less threatening to 
teachers. However ^ if used for cbmpensatibn decisions ^ student reports will 
be more threatening. , 

Professionals (such as Aieamoni 1981) urge that student rating foras be 
carefully constructed with the aid of professionals to increase reliabiiityi 

Others such as McGreai, (1983) warn against certain questions that are directly 

- - ; . - s - - - - 

critical; He prefers -wording such as "I get help when I need it," and "I feel 

tny ideas are important-*— strongly agree, agree, disagree, strongly disagree" — 



CO direct questions such as "Does the teacher know the subject matter" arid "Is 
the teacher interesting^" 

The administration of questionnaires heeds to be done in a way that 
assures ahbhymity for the individual respondents. If the questionnaire is \ 
given but tovjards the end of the year, especially after grades are given oat^ 
pupils are likely to be less concerned^, about the possibility of information 
getting back to the teacher and affecting their grades. In any case, 
repprting only group data is the proper professional practice. Individual 
questionnaires need not be signed and should not be directly handled by the 
teacher being rated. 

In sumtnary, if the ratings are obtained near the end of tKe sclK)^ 
student ratings can provide a client perspective oh dutcdmes arid prdcess 
factors — a perspective riot otherwise available, ^ind they require a miriimum of 
resources. 

The key qiiestidn is whether students are able to provide valid and impor- 
tant ratings of their teachers and whether they will do sd^ particularly in 
the context of knowing that the information will be used for helping determine 
teacher compensation. For the purposes of teacher evaluation^ the evidence 
appears to indicate that students are surprisingly objective in their ratings. 
The approach used in Seiiing (Oklahoma) with student test scores is of rele- 
vance here also. Based on interviews with students, in which a few suggested 
that some students control their answers to the test scores in order to reward 
or penalize certain teachers , sSeiling altered its procedure to disregard the 
lowest ten percent of the scores in any given year (see case report); The 
same approach could be used for student ratings. 




Parents also have a unique perspective and are also a client, alt.hdugh an 

indirect one, of school districts. Parents can be mailed or given a 

'questionnaire to.evalaate the development of their children in particular 

grades and subject matter. As with student ratings, parents can be queried 

about their perceptions of their children's learning progress and affective 

development. The jparent can also report bh some classroom process eleniehts^ 

such as homework^ discipline^ and parent-'teacher communications. 

The primary advantages of parent ratings are: 

' o Th^y 4^^^^ ^ unique perspective on their children's development, 
providing outcone information on both cognitive and affective 
:devej:opment, as well as some (but liuAlted) classroom process 
irformatibn* 



b fairly easy and. inexpensive to obtain (at least once the 

questionnaire has been developed)*^ [ g 

The primary disadvantages are: ^ : 

o Parents have a relatively limited perspective on a particular teacher's 
performance (part:icul5rly in upper grades) and toay not be able to sort 
out other factora affecting their children's development* 

^o Response^ rates to the survey may be low, causing questions as to the 
representativeness of the repsonses. 

o Teachers may feel uncomfortable with such ratings. 

The principal issue is whether parents can validly rate iiundrtant aspects 
of teacher performance. T^e research literature is qUl^e sparse bh parent 
ratings. Most of the reviews of teacher evaluation procedures do hot consider 
parent evaluations. (See^ for example^ Darlihg^Hammbhd et al . 1983; Lewis 
1982; and Peterson and Kauchaki 1982.) • 

The ERS June 1977 survey found tliat only 6 of over 1^000 school districts 
responding (les^ than 1 percent) reported use of feedback from parents as part 
of their formal evaluation of individual teacher performance. Only one of the 
school districts we examined is using parent evaluation (see Evanston, 



Illinois I case report)^ and it was hot using the feedbaclc as a formal part of 
its process of evaluating teacher performance for its merit pay plan. 

McGreal (1983) briefly looked at parent evaluations for teacher 
evaluation systems i He suggests that it is logical and fair to include 
parents as part of an overall appraisal system. McGreal ^ however (citing P. 
Abramson, "Ednews," Schdlastic-Teacher^s Edition , February 10^ 1975) ^ notes ' 
the example of the Berkeley^ California School District. The school district 
found that the parent's feedback "offers nothing that wasn't already known and 

9^ ___ 

it appeared that the most significant benefit of the program was its public 
relations value."' McGreal points out that parents may be in a special 
position to help assess puPii achievements on affective elements and cognitive 
elements as well as classroom characteristcs i 

In an informal interview we held with an elementary school psychologist, 
she indicated that parents might be asked such questions as "As you think about 
Jimmy's behavior last September as compared to now, has he become: More sure 
of himself? More sensitive to his acquaintances? More considerate? More moti- 
vated to learn? Etc." She felt that better teachers would show up better than 
poorer teachers on such questions, based on the aggregate of all the responses 
received from parents. As with student ratings ^ parent ratings oh any partic- 
ular teacher would be obtained from a substantial number of parents so that the 
aggregate rating would hot be influenced unduly by a few extreme ratings. 

Parent ratings (as with student ratings) can be obtained inexpensively i 
They require little teacher or supervisory/administrative staff time other 
than to administer the questionnaire once (or twice) during the year and to 
tabulate the ^results. 

Parents may have a somewhat limited perspective on teacher performance, 
particularly in upper grades when their children are taught by several teach- 
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ersi 'In addition, the-»teAGtr&lrs<;themselves may not perceive isuch feedback as 
being very useful. Unfortunately i as rioted earlier ^ there has riot been much 
expeirimeritatibri or research with this approach or with particular parerit 
questibririaires- We fbund no tests of the use of parent ratings as part of a 
teacher incentive plan • 

To illustrate the procedure, however, we draw from the experience of 
Evanston (Illinois) (see Evanston case report). Evanston distributes the 
questionnaires to parents through the homeroom teacher at the spring confer- 
ence with parents. The pareerxt is handed a questionnaire at the end of the 
conference and asked to fill it out before leaving (and leave it in a special 
depository) or to fill it out at home and return it to the school. A central 
office mails out questionnaires not distributed to the parents. The principal 
of the school is responsible for reviewing atti tabulating the questionnaires 
arid preparirig a report bf the schbbl results. The questibririaire iricludes 
abbut 50 data elemerits which take abbut 10-15 miriutes time fbr a parerit to 
complete. The questibnriaire asks the parerit to rate the qualicy of; teachirig 
in each of fourteen subject areas (such as mather^tics ^ science^ social 
studies^ art'^ physical education^ French, etc.). The questionnaire also in- 
cludes specific questions about parents' perceptibix bf hbmewbrk assignments^ 
discipline, and communication between the parent and teachers (it also in^. 
eludes a series of questions that assess the principal and the school as a 
whole), 

in summary, the use of parent feedback for assessing teacher performance 
probably warrants more attention than it has been receivings Tt^ however, is 
riot likely to be adequate as the sole way to evaluate teacher performance, but 

'_ V . _*_ _ 

it may add a desirable and relevant extra perspective. More experimentation 
and systematic testing is needed; 
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* Students Achievemeht Test Scores , 

The use of student test scores is a heatedly de^bated issue in teacher 
evaluation and will also be for use in teacher ihtentive plans. It seems to 
be widely accepted that it would be highly 'desirable to measure student' 
learning if the amount of learning attributable to other, hbn-teachfer, effects 
could be separated out. Since this is very difficult if not imposs-ible to do, 
use of student test scores as a primary way to assess teacher performance for 
incentive plans will remain controversial. 

Test scores^ where used^ are potentially th.i major way to measure the 
cognitive outcomes of teaching. They do riot attempt to assess affective 
outcomes. To some extent^ student learning can also be assessed by ratings by 
students and parents (as already discussed) ^ but such ratings are considerably 
more subjective and not likely to be as valid for determiriirig the agiburit of 
progress in learning as is an objective testing program. 

As stated by McGreal (1983, p. 129), the assessment of student achieve^ 
ment is very iogical but has major practical and political implications • The 
use of student testing assumes that an important function of teaching is to 
fenhance student learning (Miilman 1981 and McGreal 1983) and that teaching can 
do so. Evaluation systems based upon results say in effect "since we cannot 
prove that^ny one method, style or process of teaching is superior, ail that 
we can do is by results'V (Feldvebel 1980, p. 18^^ 

The student achievement test score issue is quite complex^ but is one 
that school districts should face up to when formulating, a teacher incentive 
program- _ / 
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The primary advantages of using student test scores are: 

& _ __ _ _ 

6 It is the r^ost objective a^jd valid way to assess the atnaunt of 
student learning for subject matter covered by the tests; ^ 

6 Similarly, it does not require tne use of subjective judgments by 
supervisors or peers ; 



o The information can be used to indicate to schpols arid their teachers 
particular, shortcomings in their studerits' lea^riirig (i.e. particular 
subject matter content that, did riot measure up -well for particular 
groups of studerits), / ^ 

The primary disadvaritages are: / • 

o Learning is affected by many other factors in addition to teaching. 
Thus ^ uriless these other factors can be explicitly considered, the 
res-ults may not adequately reflect teacher performance. Some 
research has ^ even iridi^tiated that these other factors have greater 
influence than does the teacher ^see, for example, Peterson and 
Kauchak 1982, and McGre;al 1983). 

6 The subject coverage of the available tests often is limited and 
usually does not match a school district's curricuium; thus test 
results may hot fully reflect teacher performance* 

o Teachers are likely to teach to the t^st, and because the tes^ 

subject matter is not a good match to the carricaium, w^y distort 
teaching emphasis and also may discourage creative Reaching; 



o it caa require considerable testing effort^ and resources — tl 

schools already often undertaHeccnsiderable testing. The use, of 
test scores for monetary iriceritiye plaris^ls likely to require more 
- . testing and more security cbritrdls bri the process. 

/ . _ _ . _ _ . . _ 

A major issue iri the use of student test scores to assess teacher perfor- 

Tiarice is what to do about the mariy other exterrial factors that affect student 

achievement on test scores. Such factors iriclude: * social arid ecbribmic 

characteristics (SES) bf the studerits arid their families^ studerit motivation, 

fDarerits' mbtivatiori arid expect atibris ^ classroom size, and the physical setting 

oi the school room (see for example: Millmah 1981 and McGreal 1983). 

The" cbricerri is than teachers will be rated negatively (or positively) 

because they happen to have a mix 6f pupils that is less able (or more able) 

to score well on th^ tests. Those school districts that are stable in terms 

of the characteristics of its pupj:j:_s_ ariA J?^ipse. j>u 
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may find this issue of only secondary concern (as it has been to Selling, 
Oklahoma — see case report )• In most locations j however j it will likely be ah 
important concern. Hdustdri (se? case report) and Atlanta (White £l_. 1974) 
have used statistical regression analysis to determine expected scores for'- . 
groups of students based on thje particular characteristics of those students, ' 
using such variables as the percent of students that in the schobi lunch 

program and student scores at the beginning of the year. Other variables such 
as class pupil/teacher ratios coultl also be. used in the analysis to develop an 

expected score for an individual class considering these characteristics Use 

_ _ . _ _ _ » 

of regression analysis depends on the school district having enough past data 

so there are sufficient data points to develop the equation to make the 

estimates. A drawback to this regresf^ibh analysis approach, as Houston found 
^ ______ _ '. _' _ ___• 

out, is that is is difficult for teachers (and others) to understand. Houston 

has altered its procedure to a less cdmplicated analysis of test scores — but 

in the process gave ^up adjusting the target test' scores for these variables. ' 

There are* tvjo basic choices that a school district vyariting to use test 

scores as part of the teacher incentive plan needs to make: 

- * it 

1. Which test to use, and^ 

2. What particular fonn of the test scores should be used to assess 
teacher performance, such as gain scores ^r absolute scores, 
comparison against national norms, comparison with other classes in 
the school district, and comparisons with past scores by the school; 

It is beyond the scope of this report to delve into the many complexities 
in these questions. However j we will briefly discuss a few aspects to 
illus^i^^e the issues involved. 

The choice of tests is a key issue. Almost all schools already annually 
administer standardized^ tests at least for some grades and for some subject 

BS 



matter i Probably ^the most frequent dbjectieti we found to the use of 
standardized.test scores, both in the research literature arid iri the school 
districts we examined ^ was the cbricerri by teachers arid 'admiriistrators that . 
these tests do riot adequately measure the content th^t the teachers are 
teachirigi i.e., the subject matter as defined By the school district itself. 
Singer, e£ ai . (i983), Miiiman (1981), and Glass ( 1974) all point to the 
desiraSiiity of a better match of test cdritent to district curriculum. 
Personnel in both of the ^districts we examined that used test scores as the 
ma jor' teacher perfornpnce evaluation technique .(Houston, Texas, arid Seilirig^ 
Otclahoma) expressed major concerns on the test content. 

A key preiv^quisite , therefore, for firbper use of student testirig in 
incentive plans appears to be- to obtain tests that are compatible with school 
district curriculum objectives. This means that national standardized tests 
ar^e not likely to be adequate ^ at least as they are presently constituted, 
^bth Hbust bri arid Seilirig are attempting to develop more curriculum related 
tests. This also means that districts will need tests-fox ^cfi— grade level ~ 
arid each subject. Fortunately, better matching of tests to local curriculum 
already may be a trend in /the tJnited States i 

This problem, however, poses immediate - problems for school districts 
wanting to move quickly into a teacher incentive system using student achieve- 
ment tests i At least initially these districts will have to rely on less than 
desired test coverage as is being done in Houston, Seiiing, and in Dallas' new 
plan. This also suggests that districts may be less able to depend pn cbmmer- 
ciaily available, stan4ardized achievement tests and even statewide assessment 
tests that may not adequately measure teacher instructional objectives in 
one's own school district (Millman, 1983). Iri ariy case^ mb^t such tests cover 
only some skills and not always all needed grade levels. 



Using locally developed tests has the major advantage that it avbiis the 
common criticism that teachers will teach to the tests if their evaluations 
und compensation depend on the test results (see> ^or example. Glass 1974). 
If the test content and classroom curriculum match reasonably well, th3 school 
districts would then want the teacher to "teach to the test." 

Local test develbpn^nt does not^ however, get around two other cdncerns 
sometimes raised: (1) that students will Intentionally affect their test 
scores to reward or punish their teachers ^if they kaow their teacher is being 
evaluated or paid based oh those test scores)^ J.hc <:2y l\:\t the teachers them- 
selves may become motivated to 'Improve their evalu xtiohs and cbmpensatioh by 
various undesirable devices such as not encouragiig students' full effort in 
the fall baseline test and piaci^ emphasis on the spring "post" te&t^-^r; 
actually giving inappropriate help to students in the spring tea ting. A fer 
Selling (Oklahoma) students in a survey conducted by the district after the 
third year of the plan suggested that some students might be controlling their 
answers to punish teachers. To reduce the effects of such a possibility^ the 
district how disregards the lowest 10 percent of the scores on a test (see 
case. report) . . * . - ^ 

Darlihg-^Hammbhd et al. (19835 report another problem with test content. 

\ _ _ __ -_ __ -_ - - - 

They report that some past research indicates that teacher behavior that is 

' ■ ■ ' 

heeded to increase achievement ba standardized tests is quite dissimilar to 
the behavibr that seems heeded to increase complex fbrms bf learnings includ-* 
ihg problem solving ability and creativity. Similarly ^ Centra and Pbtter 
(1980, p. 285) indicate that when the measure bf student achievement is a 
miiltiple^hbice, fact^riehted test^ in is hbt pbssible tb assess higher-order 
types of iearningi No conclusions regarding the effectiveness on teacher 
questioning at various levels should be drawn from such studies. These 
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findings seem particularly significant for the selection of the type of tissts 
that should be used; it indicates that teste are needed that cover both kinds 
of learning/teaching. If the te^ts do not^ cover higher-order learning by 
students, the^ may discourage teacher^ from trying to develop such thinking 
and reasoning and from trying to achieve desirable affective development such 
as independence, curiosity, and positive attitude toward self (Darlirig-Hammbhd 
^r-^r. , 1983, pp. 280-296). 

The second basic technical question with the use of test scores is V7hat 
• onri of the score should be used. 

Teachers' perf ormance could be evaluated in any one or more of the 

- - - . 

following ways: 

o On the ;gairi in a class' scores (e.g. average score) from the test 
results at the beginning of the school year to the end. 

b Oh the difference between the scores of the teacher's classes and 

those of similar classes (at the same grade level and subject matter) 
in the school district. 

o By comparing the class' score to a statistically deveiaped "expected" 
score (as discussed' earlier). ^ 

b ^y_^?™P?F^?i ?^^i^Y^???^ _ ^_S?i?_s^€^e or the 

?PE^?i ?5?^?_^®Y®^_^^^^i^^ against some non-statisticaiiy developed 
target for the ciass. « / 

in addition to the u^^ of a ciafes' overall average score, a district can 
also use performance indicators .expressed in the forms percentage of a class 
that achieves a certain score. This latter option will possibly be of 
particular interest to schools with students that have troubles reaching 
minimum j>rdficiency levels 

Gain scores can be calculated by cbmparihg test scores^ at the beginning 
of the school year to the end of the schbbl year such as in Seilihg. (Okla- 
hdma) — see case tepbrt — or frbm bhe year,tb the next such as in Hbus-tbh arid 
Dallas. The^brmef prbcedure requires testing studies both at the begi^ihirig 

71 ■ ' ■ - '■ 
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arid at the end of the school year. (Faii-spring testing hag the advantage 
that the cla^ composition shoaid be about the ss^tne, thus avoiding the 

possible confounding effects of comparing a somewhat altered mix of students 

._ . . ] . » 

because of changes over the summer.) \ 

The cost of student testing can become substantial ^ especially to develop 

new tests that better m^rc" the school district's curriculum than do currently 

'■^■^ 

existing of f-the-shei.r ^/ i:s. However^ much of the testing may be needed for 

other purposes as well i o that such testing is not necessarily solely for the 

inctf^ive plan. There will be costs for purchasing of materials, possibly 

special proctoring of the tests, and scoring and analysis' of test results on a 

teacher-by-teacher basis (or^ for c>r^ap incentive plans such as Houston and 

Dallas i or a schdol-by-schobl basis )i A problem also arises as to the 

frequency with which tests of a certain grade and subject would^need to be 

changed so chat teachers would not become familiar with the test questions and 

be tempted to provide coaching in advance to their classes. Neither of the 

school '^istricts we examined that were usiiig test scores as a. major part of 

their teacher performance evaluations felt this was currently a problem but 

»\. 

felt that it could possibly become one.- * . \,z_i -.:.:::::z::7z::."__j_^_ZL 

/ 

Gould classroom grades be used to evaluate teachers for incentive plans? 

Teachers ev^^luaridn researcViers ;appear to seldom raise this option. Miliman, * 

I - V 

(19b-.. p. 161) summarizes the argument against their use: differences among 

teachers and their grading practices mean that classroom grade scores would 
- --------- J - 

not likely be reliable. Since their use would mean that ^^ihe persons being 

evaluated would generate their own evaluation records, th^ir use would not " 

likely have much credibility among outsiders (e.g. the public) or even among 

teachers themselves. 

I 
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Glass (1974) discusses ah earlier suggestibh by Pbpham, McNeiii and 
Millmah to test teachers by having teachers teach a particular topic to an un- 



familiar ^ randomly assigned group of pupils for perhaps 30-60 minutes — with 
only a few hoars or a day's notice and then to measure pupil test scores. 
This conceptually would provide a rigorous evaluation of the teacher's pe: 
formance. However, this approach has major problems. It would test only 
certain types of instructional ability and hot longer term teaching ability. 
It would also penalize teachers who need time to prepare and deliver subject 
matter. It also would require considerable preparation and administration 
cime to be able to cover all teachers in all fields^ (Popham later, 1974, 
/— recbmmehded against the wide use of this approach.^ . 

To summarize^ tne use bi test scores h»s considerable appeal, especially 
if (a) tests can be ueveloped that are compatible with school district 
curriculuui objectives and (b) if seme consideration can be designed into the 
procedures .to consider and adjnst for possible differences in student 
cbmpbsition f rom bne class or one 'school to another (e.g. so that teachers of 
classes with more •'difficult" pupils will not be penalized) — such as by using 
statistical analysis to adjust for differences. / 

Test score results have the great virtues of being objective and directly 
^ facing up to the chief objective of schdols — student learning. They appear to 
, have a higher face validity than do ratings by supervisors or peers .(the 

Tresearch on supervisory and peer ratings raises tod many questions about the 
^ adequacy of these ratings as a proxy f b * student learning/achievement). 

The approach does require substantial care and cost in deVelopIag aopib- 
priate tests and administering^' inbriitbrihg, arhd analyzing them/ Opponents tb 
their use are concerned about the lack bf cbrrelatibn between teacher effort 
and student learnings pfeintihg tb the research that indicates bther family and 
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rabtivationai factors have considerabiy greater infiaence on .^>*adent learning 
than does the ie^her; However, if teachers do not have an appreciable effect 
on learning, Chen pay-f or-perf ormsitice plans may not make mtich sense anyway, 
if they can have some tangible effect, as most districts will undoubtedly 
agree, then school districts heed to decide whether the district's philosophy 
is to reward teachers merely for good effort ^trying hard) or whether to focus 
on results^ even if the results are not always due to the teachers themselves. 

Use of School District Record Data . 

Some school record data can be used to assess aspects of teacher 

performance that school districts may wish to include in performance 

^ , ^_ . 

evaluations for*teacher incentive pianSi Prime examples are teacher and 

student attendances Houston, for example, requires that teachers have no nidre 

than five absences in order to be eligible for any of its several bonus awards 

(see case report). In addition, one of its bonus awards is for teachers that 

have fewer absences than this target. Dallas' hew plan considers both teacher 

and student absences (Dallas 1983). 

Teacher attendance may hot be a problem in many school districts ^ but 
where it is (and leads co substantial added costs for substitute teachers or 
disruptibh), it could be factored into the incentive plans. 

The relation of teacher b'ehavibr to student attendance is not clear but 
student attendance (or rather unexcused student absences) can be considered as 
an indicator of student attitude and thus possibly is affected by that 
teacher. As with test scores, hdv/ever^ consideration of class "difficulty" is 
desirable to provide fair teacher evaluations. Teachers with students whose 
background characteristics are more liable to absenteeism should have lower 
attendance targets. : . . * . 
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Hdustdh ppihts but that it had ah abseritiaeisin prbblemj having a specific 
bonus related to absenteeism helped it meet an Explicit district objectives 
In other school districnTs^where teacher or student absenteeism is not a 
problem, these may not need to be included in the ^.ncentive plani 

Other record data elements that can be used are external awards for stu- 
dents and college acceptances. This information will, however, be of limited 
use for assessing individual teachers but would be more useful for evaluating 
individual schools as part of a group iriceritiye plan. It has another major 
drawback; this irif orTnatibn will likely reflect the backi^rburids arid iriherent 
abilities bf the students mbre thari teacher capabilities. But^ nevertheless^ 
teachers prbbably have sbme irif luerice bver these arid trarids bver time could be 
indicative bf teacher perfbrmarice. 

i The primary advantages of school record data are: 

o It is objective, arid probably accurate^ information. 

b Since it is already avaiiabie, it would likely cost little to use for 
incentive plans. 



Its primary disadvantage, is : 

o it is avaiiabie only for a few elements relevant for teacher perfor- 
mance assessments and these elements are insufficient by themselves 
to provide adequate teacher assessments. ^ ' 

We found no research information on the use of this type of teacher 

evaluation. The information from Houston, the one merit plan we examine^ with 

experience on rewards directly related to record data, indicated that the plkn 

had succeeded in obtaining significant reductions in teacher absenteeisti. 

I' 

Hbwever^ whether this reduction has affected studerit learriirig or tb what 

exterit it has reduced the schobl district's cbst is riot kribwri. 



Cdinparlsdn of Actual Performance to Targeted Performance * ^ . 

A number of school systems have procedures for establishing objectives 
for individual teachers at the beginning of the school year with some type of 
exaciination of accbmplishmeats against these objectives at -the end of the 
school yeari Accompirishment of pre-set objectives can be used as a major 
teacher evaluation approach. In Chapter 7 we discuss this type of plan as an 
option for a non-monetary incentive program. In this section, we focus oti it 
as an option for monetary incentive plans. 

In a teacher merit pay plan, individual teachers (or groups of teachers 
for a ^rdup plan) at the. beginning of the school year would each e**!:ablish 
with their supervisor a set of perfbrmarice targets fbr the schbbl year. At 
the end of the year^ the actual^ results wbuld be compared tb the targets. 
Those achieving or exceeding their objectives would be eligible for merit 
pay. "This type of "perf onaartce-by-pb jectives" (or "Mahagemeht-By'--0b5ectives") 
approach is hot by itself a complete measurement procedure. The specific data 
collection procedures that wou^d be required to determine the extent to which"^ 



the targets have been met will: depend on the objectives. One of' more of the 
previous performance evaluation procedures would be needed °to measure progress 

towards the specified objectives (eig. if an objeccive is to have students 

/ . 

_ __ . _ ; : ^ _ . 1 , . > 

reach certain proficiency Jevels, then testing would be needed) • v 

The- primary advantages o£ an actual vsi tatrget achievement approacffare: 

b Teachers have an important role in establishing, with their supervi- 
sor, their objectives fou^the school year; thus, this is a highly 
participative prbcedure with an extra opportunity to motivate 
part j cipating teachers. , " 

d The prbcedure is relatively simple and straightforward in concept. 



V 

o 



Since targets can be tailored fdr each teacher, the procedures 
provide the bppbrtuhity to adjust for individual differences in 
classroom cbmpbsitibn^ student difficulty^ and bther special circum- 
stahces*i e.g. a teacher with a class having low past test scbres" 
could set a lov/er class test-scbre target tilari . a teacher' with a cla^s 
•that had; higher testscbres. 

' ^ 76 
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the primary disadvantages are: 

o the targets set by different teachers are likely to vary considerably 
as to importance to the district arid difficulty, thus causing 
problems in the comparability of awards between teachers. 

o Teachers and school administrators heed training in the 

identification of - appropriate targets. The process is not a familiar 
one for teachirfg staff^hd hot as easy to carry but successfully as 
may appear at first. The^e is a tendency for targets to become 
relatively insignificant and easily achieved. 

b The targets teachers can set are limited by the capability of the 
district to measure accurately the degree of target achievement i 
This procedure is hbt likely tb reduce measureSent problems if 
meaningful bb jectives/targets are set by teachers^ 

A mor.e thorbugh discussion of the advantages and problems with target- 

< 

setting procedures — for non-monetary plans — is presented in Chapter 7. Most 
of those points are also applicable to monetary plans. 

Three of the school ^stricts we examined that had monetary incentive 
'plans also had some form of objective-setting. Round Valley CCalif brhia) ^ 
Evans ton (Illinois), and Lebanon (Connecticut). The latter incentive plan has 
been: discontinued. Round Valley's principals consider achievement of the 
teacher's objective along with classroom dbservatidns to determine the point 
score for 35 percent of the teacher's overall merit pay rating. Evans tbh 
principals also nsider achievement of bbjectives along with, classrbbm 
observations tb die termine each teacher's overall perf brmaiice level.. In 
Evanstbhi teachers amst hav^ phe studeht-prbgress bbjective bf perhaps two to 
three targets. Tlife district feels that having mbre targetf v/ill lead tb 
dilutibh 'of the tea.cher's effbrts and perhaps bverwheim them. Lebahqh 
.teachers were required to select one result objective (on student progress) _ 
and one process bbjective. -Teaches had to Submit reports on their progress 
towards their bbjectives five times during the year. (The district reported 
that the paperwork associated with the various reports became a significant 
burden, probably contributing to the plan's ultimate demise.) in none of 
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these cases I however > was the targeting process the central part of the 
evaluJatibh. 

The data available oa these plans do not indicate how successful these 
aspects of their plans have been. Nor did we find any past research or 
experimentation on the use of targeting for teacher mbnetary incentive 
plans. (As Chapter 7 indicates^ there is ^}-r^t no systeuitically cbllecCed 
evidence on their use in nonmonetary educational system plans either^) : 

A number of school districts are using some form of teacher objective- 
setting in part because of State laws (e^gi in California and New Jersey) i 
Thus far, however^ we suspect most of the school districts have not introduced 
full-fledged targ'^^t-setting procedures with specific measurabie objectives— 

i ^ 

see Chapter 7i 

A vay to overcome one of the major problems with this prbcedures that of 
having comparable targets across teachers^ is suggested by Round Valley's plan. 
Round Valley uses a procedure on other parts of its merit pay plan that might 
well be adapted to the use of perfbrmarice targets: Have a central comprised of 
admihistratbrs ahd^ pbssibly^ teachers assess the individual targets at the 
beginning bf the schoo^ year as to their difficulty and ,imp6rtahce\tb thr 
schbbl district and decide on the maximum number of "points" that each target 
warranted. Preferably, also, at that time specify how m&ny points would be 
awarded for different levels of achievement for each target. (Round Valley set 



maximum points for. two components of its plan: special r:eaching initiative , 

J . o_ ■_ ^ _ 

projects and teacher cooperatidil projects — see case reports) 
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In suEomary, this procedure is quite attractive in petYmitting teachers to 
set thei]? bwti target objectives (with supervisory approval), thus giving 

teachers a substantial role in the procedure. There are a number of problems^* 

- : - J 

however, especially the difficulties teachers have setting meaningful targets. ^ 
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and the problem of different teachers setting targets of differing impbitarice 
arid difficulty. These problems need to be bvercdme for the procedure to be 
useful, otherwise there will be complaints from teachers not receivirig merit 
awards who believe that other teachers set easier and less important targets. 
There has been little experience with this typej of procedure. More testing and 
experimehtatibh seems needed before the approach should be widely used* 



The Use of a Cornhf natinn nf Tpg^h^T- Pgrf nrTngncp* ^^glggtiOTi Procedures i 



A major option sdhooi districts should consider is the use of a combina- 
tion of these approaches, in effect, many* of t'he incentive 'plans we examined 
did use combinations. Houston, which focuses primarily oti test scores,' also 
uses, attendance records , and to determine' eligibility for the bdriuses, uses 
subjective supervisory evaluatidri ratings. Bryan (Texas) ^ though etnphasi''.irig 

supervisory classrbbm bbserva tibris ^ alsb ericburages principals tb use bther 

- i _ _ _ _ _ _ ^ 

infbrmatibn that is available^ such as teacher absenteeism data and informal- 

feedback frbm parents and students. Evahstor. (Illinois) uses supervisory 
classroom bbservatibhs ^ performance against targets ^ and (informally) parent 
ratings. Plans that involve comparisoii of performance targets against actual 
achievements, such as those of Evans ton and Round Valley (California)^ gener- 
ally will heed to use a variety of procedures (such as test scones, record 
data^ and supervisory evaluations) — depending *on the particuiS- objectives 
targeted by the teachers i Supervisory ratings, especiaiiy if based on some 
systematic procedure for classroom observations, have the aavantage of perralt- 
ting the district to encourage specific 'teacher improvements in classroom 
practices, which is much less possible if the district focuses solely on stu- 
dent test scores. On the other hand, if the procedures focus solely on super- 
visory (or peer) ratings, this loses any direct focus on student outcomes^ 
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the performance eva Ilia tid.ri procedures for merit pay plans could well 
a cdiiibination of these measurement approaches ^ such as ce*^ c scores to 
determine student learning resultSi pe.Thaps student (arid even parent^ ratings 
to provide a client-oriented perspective^ and supervisory icitings to identify 
more detailed ways to iiQprove teacher performance i 

A fin^^i comment; teachers should know as early as possible, by the 
beginning of the school year if not soonerv what specifically they will be 
rewarded for — so they can develop their plans to meet those requirements^ The 
motivational value of an incentive plan will diminish 'to the extent' teachers do 
riot Rridw clearly what the requiremerits for merit av7ards a^fe arid have enough 
time to adjust their actibris and behaviors to meet thos^ requirements. 

• . '■ 

how Shou ld Teacher Evaluatibris Be Linked to Specific Award 'Amounts? 

=- 7 ■ : 

Once a school district has decided ori: which specific procedures it will 
use tb evaluate teach'". r :>rmance each year^ there is another crucial choice 
to be made# How sboulr. tut* ratings or scores be used t6r-4etennine who will be 
given awards and what size each award should be? Even the best . performance 
evaluation can come to nought if this linkage is poorly handled* 

The major concerns here are thatt 

' . .. <^ 

o The relationship between the teacher performance, .evaluation measure- 
^ ments (whether student test stores, supervisor;^ ratings , etc.) and the 
awards be clear and understandable to participating teachers and 
administrators- — so they^ know what performance is needed to receive 
what rewards ; 

_ t 
- : ^ 

o This linkage prodess not introduce a major new element of . 
subjectivity; and ^ 

o The linkage is perceived as fair by the participants* 

The Urban Institute fduiid this step to cause major problens in local * 
government mariagerial iriceritive plans (Hatfy et al * 1981), Some plaris^ even ^, 
though havirig specific manager evaluation procedures ^ used a highly subjective 



: and the problem of different teachers setting targets f' differing .ixnpdrtahce 
and difficulty. These problems ne^d to "te overcome for the procedure to be 
useful i otherwise there will be complaints from teachers not sreceivirig merit 
awards who believe that othei teachers set easier and less important targets. 
There has been little experience vith- this type of procedure. Sore testing and 
experimehtation seems needed before the approach should be widely usedi 

The U s e of a Comb i n a t i on of Te a ch e r Perfom i ance Evalu a tion Procedures > 
, A major -option school districts should consider is the use of a combina- 
tion of these approaches. In effect, many of the^tincentive plans we examined 
did use combinations . Houston, which focuses pri-marily on test scores, also 
uses attendance records, arid to determine eligibility for the JjbnUses, uses 
subjective supervisory evaluation ratings. Bryan (Texas), though emphasizing 
"supervisory classroom obseryatiohs ^ aJSv encourages principals to use other 
information that is available^ such as teacher absenteeism data and informal 
feedback f rom ipa^ents and students^ Evanston ( Illinois) ^ses super\ -y 
classroom observations ^ performance against targets ^ and (infdnhally^ parent 
ratings. Plans that involve comparison of performance targets against actual 
achievements, such as those of. Evanston and Round Valley (GSi 
ally will need to use a variety of procedures (such as test scores, record 
data, and supervisory evaluations) — depending on the particular objectives 
targeted by the teachers. Supervisory ratings , especially, if based on some 
systematic procedure^ for classroom observations, have the advantage of permit- . 
ting the district to encourage specific teacher im^tdvemehts in classroom 
practices, which is mUch less pdsi^ible if- the district focuses solely on stu- 
dent' test scoves. On the other hand, if the procedure-s focus solely on super- 
visory (or peer) ratings, this loses any d "mcus on student dutcdmes. 



and vague procedure for determihi hg the specific merit increases; causing 
confusion over what performance : needed to receive what merit pay; 

A school district cr^ use a pre-rpecif ied procedure that 'specif ies ahead 
of time what amount of merit pay will be |iven for what specific performance • 
Both Houston's and Seilirig's (Oklahoma) plans do this with their detailed 
Specification of who gets what for what performance (see case reports). Note 
that these are also the two plans with a major focus on test scores. 

Plans with a quota system (such as Bryan, Texas— -see case report) or with 

a budget limit on the total dollars that can be distributed (such as Penn 

Manor, Eennsylvania—see case report) do not guarantee that a certain level 'of • 
" _______ ' ' js . 

performance during the year will yield a specific size award; Ifiis is probably 

a handicap for incentive plans by not allowing teachers to be assured they will 

receive an award of a certain siz^ gr even any award, for specific 

achievements; 

Some jurisdictions have numerical point systems often associated with 

-1 - - - . - - - ♦ - c ■ 

Supervisory rating systems. Teacher evaluatdrs determine the number of points 
the teacher receives oh each of a number of characteristics. The range of pos- 
sible jpbirits is pre-determihed . These points u&ually< reflect the relative 

........ ^ ^ . . , , ^ ■ 

weights (i.e. impbrtahce) for each characteristic. For example Ladue (Mis- 
souri) gracfes each teacher from 0-15. Each poirit is icurrehtly worth $300. 
-4lound- Valley ^CCalifdrnia) also has a point s]^ stem v/ith'the number of points 
received by a particular teacher automatically determining the amount of.- award 
the teacher receives. .(Currently each point ia worth $280, with a maximum of 
ten points divided among three criteria--see Round Valley case report.) 
Tredyffrin's (Pennsylvania) supervisory evaluation' system awarded from 0-i2 
points on each of a number • of characteristics. At the end of the school year, 
the total number of points received by ail teachers was ad^ed up and divided. 
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into the total aindiint allbLated by the school district to deterirdne the dollar 
value of ^ach point (se^ case report J. 

Complete pre>^specif icatibh of what level of performance will receive what 
level of award has many advantages. This, however, will not liiceiy be every 
school district's preference. Districts, however, should ^attempt to be as 
ciear as possible on the linkage between perfdrtnance levels and awards to 
maximize the motivat : onr 1 value of the awards • 
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; CHAPTER b 

Other Incentive Plan Design Issues 
in this chapter we address the following issues: 

d To what extent can and should the teacher evaluation procedure also 
be used to identify ways to improve perf brmahce? 

6 To what exteat should information on who receives awards be 
promulgated? 

6 To what extent should teachers and their associations participate in 

the design and implementation of the plan? 

o How much advance planning and preparation tiita is needed? 

o Should there be an appeal process? 

To What Extent Can a n d S hould the Teacher Evaluation Procedure Al so^€-Use4 to 
Identify Ways to Impr o ve Tea c^^^^^rf armance? 

Ideally, the procedui^es used to evaluate teacher performance lot che 
■purpc ^-s of determining cdmpensatl'^ " would be equally useful for giving guid- 
ance to achers as to ways to iib* e their future performance. Unfortu- 
nately , it is likely to be dif f i^. .it l- achieve both of these purposes with 
the same procedures* 

Sbme^-teac^ ir evaluation experts believe th^re is a direct conflict 

between evaltxatibns to determine teacher status iuestibhs such as_ cbmpehsatioh 
("suDunative" evaluation) and evaluations to help identlj.y what needs to be 
done to improve performance ("fc.ritive" evaluation)* For example. Darling- 
Hammo'd et al» (198.1^ identifloH the fbllb'^ing cbnflicts betwe*»h thase types 
of evaluations information affecting ^oh status^ s,ach as pay, \:ixi generate 
anxieties among teachers and make them less amenable to constrnjctive discus- 
sion about their performance; the need for objective infom^ation for determin- 
ing specx;?l compensatioxi — inf osnnation that is externally defensible — may 
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descrij ive , and illuminates sdurcfes of difficulty as well ao viable courses, 
for change" j^p. 303). ERS (1978) points but that there can be a problem 
between the two roles of adminis tratci's ^ that of jnaklhg judgments (e.g. as to 
merit pay) ana that of counseling (helping teachers imprc/.ve th^ir future 
perf oirmance) ^ 

On the other' hand, Scriven (in M1:iifflan» 1981) argues that "valid summa- 
tive cype evaluation is the essential basis for recommending and detecting 
improvement" (p. 267). He points out thac summative ;vnf ormatioti can provide 
clues as to problems needing attention even if Euch inforsiation is. not as rich 
as that- provided by other types of procedures. 

A school district's desires regarding these two roles d^n directly and 
greatly affect the choice of teacher perfbrmahce evaluation procedures. 

Of the pr6c<2duras discussed in Chapter 5 i supervisory and peer rating 
procedures are most directly relevant to the role of assisting teachers to 
improve their performance. The key q'. est ion regarding these prrcedures is 
whether the eiements examined for the ratings are sufficiently valid to lead 
to t^ie impacts the school district wants. There are the dangers noted In 
Chapter 5i For example, in our iaformai discussions with teaching perronnel^ 
they frequently pointed to situatiotis where a^ teacher who might not h/zve had 
ch^r^;;acteristics usually associated with desirable teaching nevertheless was 
quite effective in encouraging student learning, at least for certain types of 
pupils. ^Nevertheless, most administrators and teache is believe that they can 
usually ideittify good and poor teaching practices and thus can improve teacher 

.^./ . . . . . ._ ._ 

behavior in the classroom even if it does not clearly le *^ to imDioved stuaent 
ieaxnln^. . ^ ^ 

Student test Suores are <^eherally perceived as iibt prbvidihg irtf brmat'idn 
to help, iciprove perfbrmahce. It sisems clear that procedures th^^ use only 



71 

gross student test scores will be the least useful approach for helping 
teachers improve their performance (see Singer et al « , 1983) « Howeveri as 
indicated earlier by Sciriveh^ even test scores can--and probably 6hould~be 
used to provide clues to learning problem areas i This will depend on 
examining the test scores systematically to identify problem areas for the 
teachers. This should become considerably more useful as more testing is 
adapted to the local curriculuir.» Ttie need here is to identify the subject 
matter for tei=;t questions ^dn which certain students had particular problems, 
and to convey that information to their teachers as guidance as to whiere 
imprbvc'nierit is needed in future classes. This process probably requires doing 
jidre analysis of test results for individual teachers than school districts 
are accustomed to doing. Even in the school districts currently using test 
scores for their incentive plans ^ we did not fiVid this type of feedback being 
providad to teachers in i detailed^ organized manner. We note that such 
analysis seems considerably more practical for e^eh smali school districts 
with cie advent of microcomputv^rs i 

As discussed in Chapter 5, student and parent ratings, if used as part of 
the district's incentive plan, can be used r.o assess both "results" and 

i ■ 

"processi" These procedures appear to have bean used by schools in the past 
mojr^rly as feedback to indj.vidual r achers to encourage the teacher to identify 
sphere improvefnents are needed. Rarely have they been used for evaluating 
teachers. For incentive, plans, the key factor is the choice di ' particular 
topics to t?e included in the que / tidnriairp For these ques tldnnaires , the 
ir.clusibh of both butcbme-oriented questions and specific process qiiestidns, 
as is often dbhei can provide both evaluative data and feedback helpful to 
teachers. For stu<ient and parent questionnaires ^ strengthening of ouestibns 
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better able to assesss changes both in learning, and in affective eletnentis df 
studeht achievement such as self -respect and Ability to get along with others. 

The ihcliis-Loa df some farm of cbns,tructive feedback to teachers as part 
of the merit plan process is often advocated, (e.g. £RS| 1979> ahci Cbngres- 
iiibnal Task Forct on Merit Pay^ 1983). The current Florida Teacher Evaluation 
pr6j':ict is developing two teacher assessment instruments, one for formative 

! 

purposes and one for sumtuative (Coalition for the DevelopSerit of -^he Florida ^ 
farfonsance Evaluation System, 1983) i McGreai (1983) points out that teacher 
evaiuat^iohs should help improve teacher performanpe ap well as provide a basis 
for rewarding superior perjormancei 

On occasion, the teach^^.r evaluation process, with its improvetaet\t- 
oriented components, hat; been kept completely separate froii* iiierit pay or per- 
f drliiar!.:e-by?-db3ectives plalis^ (TUIs has dccurred t'requently in dther state 
Hhd local gbvernment, hbh-educawlbhal agencies- where the antiual ^xnoLbyee per- 
fbrmahce a|>praisal process has imtil recent years been usually tfa'^ed as a 
separate^ independent effcrt—see Greiher ^1. ^ 1981.) This practice has 
the danger of prbvidihg conflicting messages to employees. Mbst teacher mone- 
tary incentive plans we examined either make the teacher evaluatlbh ^ central 
part of the plan or at least require that the teacher evaluations b'i adequate 
to serve as a basis for merit pay (see^ for example, the Fouston ^ase report). 

MG3t- school districts that use supervisory or peer ratings require •:on- 

fererres between evaluators and teachers shortly atter classroom observations. 

«• _ _ — _ _ — ^ 

(Teacher self-evaluations ar* ^soffletimes also required, but: t.his is not a topic 

that seetrs relevant to en examination of incentive plansi) 

Overall, it seems j-^irable for a school district ^itig an incentive plan 

td have prdceQi -^.r that cdmbine >*ct't pi rpoces, providing some substantial ' 

''^cUiBS td teacheis : . r iiaprdvement: as well as irif dtm^tiori for eva uating the . 
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gross stadent test scores wili be the ieast asef ui approach for helping 

teachers improve their perfoxTsance (see Singer ^J^» , 1983). However, as 

indicated earlier by Scriven, even test scores cr^n—and probably should — be 

used to provide clues to le a rhiiig. problem areas. This will depend qh 

examining the test scores systematically to identify problem areas for the 

teachers. This should become considerably more useful as more testing is 

adapted to the local curriculum. The need here is to ideg^ify the subject 

matter for test questions oh which certain students had particular problems, 

ana to rcorvey that information to their teatlvsrs as guidance as to where 

improveraent is needed in future classes. > This process ptobabiy requires doing 

more analysis of test results for individual teachers than school district!; 

are accustomed to doings Even i.: the school districts currently using test 

scores for their incentive plans, we did not find this type of feedback bejing 

provided to teachers in a detailed, organized manner. We note that such 

analysis seems considerably isore pr-* -t-ical for even small school districts 

Wilh e advent of tnlcrde tors* 

As discussed in Chapter 5, student a parent rarittgs. If used as part of 

t __ _ .. ^ 

the district's incentive rl^^> used to assess both ''results'' '^nd 

_ _ _ ' ' 

"process." These procedures appear to have been used by schools in the past 

mostly as feedback to individual teachers to encourage the teacher to identlly 

where i»hvrovemehts are hi^ded. Elarely have chey been used for evaluating 

c ^achers. For incentive plahs^ the key factor is the choice of the parti-^ular 

t upi-^s to be included in the quf^stibhnai'-^. For these tjues tibiihaircsi the _ 

inclusion of both outcomc-^riehced questions and specif ic. prefers questions^ 

as is often doue, can provide both evaluative data and- feedback helpful to 

teachersi For student and parent o- i tonnaires, strengthenin . queEttons 
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teachers' performances The main probiem here is that mixing the two purposes 
can require coiu»iderabie extra paperworks Bat one way or another, the teacher 
evaluation process should be coinpatible with the incentive piani 

. To What Extent Should Inforaatidn on Who Rec^i^ve^s-^ew ards B e-Jto? mul^ated? 

Officiali public recognition of employees who have performed particularly 
well in a given year is common iu »bme pa^rtj^s of the private sector and to some 
extent in the public sector. Such re:c6gniti6h itself is believed by some to 
be a motivator. Ofteh^ however^ plans involving such widespread public reccg-- 
nition are limited to a small number of employees. In the teaching profession 
this could be a teacher-of-the-year award for a school district or state. 

However, tho promulgation of the names of teacher; who have won merit 

awards appears to have had more negative consi:quences than positive ohes^ at 

least in the cases we examined s Ttiere. are two major concer ^ parents 

may want to move their children into classes taught by rsacr lo havfi 

received laerit pay, and similarly they may be unhappy at having cheir children 

in classes->^taughr by teachers who have not received the awards. This could, 

for example, '^require schocj districts to devise a procedure for assi^ing 

students to teachers in souie way that parents perceive as being equitable 

(Cbhgressibnal R^se^rch Servi/- , 1983). The CRS report also indicated ih^^ 

the problem might hot be as great at the secondary level as for elementary 

schools because at the h^gh school levels studehcs may spend only one pdridd 

per day with a particular teacher. * • 

The second major concern is chat thoi*e teachers who do not receive 

'_ . _ - ^ _ _ 

^ award may suffer from poor morale and loss of .;elf esteem and prestige. Tuis . 

is of particular importance, since iQOSt teachers are quite dedicated to their 

profession of helping their IgJrudentSi <We have already noted this concern in 



approaches would reduce tho embarrassiiVc ri<il;iJ.'j^l Le^chers tot receiving 
awardi. ) " 

Lebahbh (Ccririect icut) did riot prbmuigJ e ui«i,eSj but Iru c rmatxor* on them 
leaked out desp"^ e a formal agreemeric ch tiui T^^acher's /issdciatibri dri cdrif.i- 
dentiai'ity. A list came out in a matt'er cf days after the awards were made^ 
apparently from unhappy teachers (see Lebanon csae report). This caused major 
problems with both parents and teachers^ This probJem -led in part to the 
considerable dppojsition that developed toward the p-i-an and its eventual 
terminatidri. Round Valley (California) has a strict confidentiality policy to 
avoid the terisibris caused by teacher jealousy, bitterness, and humiliation 
tsee Rburid Valley case report arid Burfce^ 1982). Catalina Foothills (Arizona) 

i ' ; 

had mixed feelirigs on this; sdme teachisrs in drie of ^ surveys felt that 
recipients shbuld be publicized whilo bthers felt ^Id damage the morale 

of hbnrecipiehts. Low visibility has^ hbwever, beeri Catalina's preference 
(see Catalina case report and Frase^ 1982). Periri Manor (Pennsylvania) made 
public the names of award recipients. Sbme teachers objected, arid the 
publicity reportedly hurt some teachers in th*> eyes of family and commuriity. 
School admti istrators, however, feel that idehtif icatibri is irievitable arid 
have nor changed their practice of releasing the names. (The number of awards 
each year is staall, rbout 25 or percent of ail teachers—see Penn Manor, 
case report.) A los Angeles 1±m&u^ article on the San Marino (Calif brnia) 
merit pay plan rejpjrted rhat although teacheri voted the plan out •af tar 25 
yiars^ the ndmiriisuratdrs attributed part df the plan's long duration to their 
ability td keep cdrif ideritr.-tl the award recir^'^nts (see also 3avage, 1983). 
Ladue's (Missouri) pdlicy is td keep the information confidential. it feels 
that salary informacion should be treated the same as compensation for ;*.:^due's 
dbctbrs, lawyers^ arid cbrpbrate executives. Schbbl bfficiiils do riot prcvide 
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iiifdrritidh about specif Ic salaries , although they must report them to the 
state (s^e U due case report and Miami Herald > August 8^ 1983^. Bryan (Texas) 
provides its bonuses qu. uly and without publicity , but teachers can go to the 
principal and see the list of bonus winners (see case report).. 

Thus, low visibility is usually the preferred approach. Avoiding morale 
problems of non-recipients is the primary factors (As discussed earlier, this 
CO ^ern and that of encouraging constructive cooperation is also a reason for 
the attractiveness to some school districts of making plans voluntary and 
using group, as well as individual, incentives.) ^ ^ 

To— W^^a4:^ £3U:^nt Should Teachers and Teacher Associations Participate in the 
Design and L Jlementatidrt of the Plan? 

. — : r .- ' 

/ School districts, anxious to move qu^-ckly into incentive plans ^ tend to 
reduce the amount of participatibh by <:eachers and their assbc\atldh« How- 
ever, a cbminbh .theme in incentive plan problems has dfteii been the lack of 
participatibh by those affected by the p3^,h. Ti\xs was a majdjr finding bf the 
Urban Institute's earlier work on incentive plans ror managerial and non- ^ 

•: ■ V • 

managerial employees in a variety of s "at?- inad- loc«i: gcvicnment non- 
educational agencies (flatry et al p lo" : a.ad Greiner ^ al.'i J.98J). JHe 
(Jovernor of Tennessee had severe problems in geti:ing iegisfation for his plan 

passed^ partly due to lack of participation by the teachers in forrauiattng the 

- J _^ y ^ 

plan. The Gongrescionai Task Force on Merit Pay (1983) recommended that educa- 
tional agencies involve teachers and administrators as well as the commuiiity 
in establishing the evaluation criteria and benefits for a .iKsrit -i^ay plan so 
that teachers as wel as school administrators and the board of education 
become firmly r dtiittii tted » '^cGre.-: i |J:^83) emphasized tue impdrtaticr* of a 
feeling of c /her^hip by teachers if they are to bt: i^illlng td change arid 



evaluation procedures'^ they also are relevant to incentive piansi The 
Sovefno-p^ "Pocket Guide" (1983) says that teachers and their organizatioris 
"must be consulted and involved in developing the prograrj but opposition from 
teacher unions can make this difficult" (pib). , The Superihtehdeht .ebanbh 
Connecticut reported that lack of participatioh by teachers and .'u' ^^ebcia' 
tibh in the develbpmeht bf Lebanon's merit pay plan severely handicapped its 
acceptance (the plan was dropped afte^- : years--see Lebanon case re-port). 

Teacher participation is probably ji : . ilarly important for developing 
the teacher performance evaluation'^prbce ..zes. For example, Shaw (1973) sug- 
gests thar participation of the teachers Iv the development of the question- 
naire is one oi the best ways to reduce the concerns of teachers abdut student 
ratings i Most of the jurisdictions with which we spoke enjphasized the use of 
' teacher inputs in the development of the 'zeacher evaluation process and in 
reviews of the incentive plan after it had been implemented; In many school 
districts, part..cipation is required by the legal agreement with the teachers' 
association; 

Par : icipatibh in the overall design bi the plan appears much less cbmmbh- 
place. Superihtehdehts and their staffs; ^ sbmetimes with cutside advisors such 
as frbm r.hiyersiti'es^ Ixave usuaJly developed the overall design^ 

After intrbdnctibn of the- pl&n^ many of the school districts ve examined 
used some form bf feedback from teachers- r.o review the plan and develop 
mbdlf icatibhs. For -rample^ Catalina surveyed ^. portion of its teachers after 
the first yeari Other school districts have used teacner councils to obtain 
feedback, auch as Sviiiing, Gkiahoma, which has'^a teacher advisory council that 
periodi£»li^ suggests modification to tht plan* liadue,- Miss^suri has teachers 

n its tandifig "Cdmi^^''. ttee bh. Ev?luk<"idh" that cbntxraw iy reviews the plan 
eud recommeticis changes; ' ^ 
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-Proposals by the Governors of North Cardliria arid AriziDria are more far*^ 

reaching; they would require approval of a plan by <^ majority of teachers in a 

scu/ol district ( Public Admlnis tratibn Times. ^ September 1* 1983). 'ir-^y- 

-opbsal ^ • however , would probably meet with opposition from many of the 

^hoL X :istilct adujinistrators with whom we spoke who beiic^rs that wtiile 

participation is highly desirable, the final responsibiiicy anS decision 

should be that of the administ ration and school board; 

Our examination of experiencey in both educational and non-eduC4^tional 

settings indicates that the more the participation, the longer it takes to 

develop and impl^ent a plan; Hc^^s^er, tUest same experiences also indicate 

that agencies that have rushed into sue", plans ^ without adequate 

participation, will likely face major negative consequences later. 

> 

» 

How Much- Advance Planning and Preparati o n Time Is Needed to Prepare a Plan? 

t - _ _ _ - _ ^ ' _ _ _ " " _ _ ~ ^ ' _ 

In the previous section^ we noted that there often are problems when ah 

hey at.tempLts to rush into ah i.-icentive plah (see^ for example^ Hatry ^t_ 

al , ^ 1982). Ladue admiuistrators warn -(^aihst mbvihg tod quickly to force 

merit pay oh teachers wxthout listening to their concerns and addressing ^nair 

needs (• ee badue case, report). Other adisinietrators also warn ?:hac rushing 

will diminish chances of success (Cramer^ 1983) i Kz: iaraazoo's plan was innro- 

duned in a few jDonths, apparently too qiiickiy (Dorema^y 1932). The tebinon 

V? jnuac' i-iit) superintendent indicated that the district riad a major problem 

trying to develop a plan to start* in Septembei: dtev gaining appros^ai for it 

onlyo in April; " 

• . ♦ . . ' . ... 

In addition to needing time to obtain teacher* participation (and if pbs~ 

si.ble, their suppbic^i time Is nreded to develop ^h .?1equate tt^apher perf->r- 

_ - _ _ ^> 

mance evaluatioh process. A. key factor is whether the district already I ^s 
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are sufficiently valid to use as a basis for determining who should receive 
mer^t pay and how much. Cataliha Foothills (Arizona) was fortunate In having 
a teicher evaluation instrument already under deveiopment that was introduced 
at the same time as the merit pay plan. The instrument had been developed 
jointly by teachers^ administrators, and the state university. Other'' school 
districts will not be so fortunate and will likely need to develop such proce- 
dures. The State of Florida is taking m^ny months to develop and test 
appropriate teacher assessment instruments. - 

School districts that focus on student test scbrei ^e likely to have an 
even greater prbb.^em* unless, like DallSs and Houston^ they al^^eady have ah ex- 

c - ' 

tensive data bank and testing capability that permit them to quickly de- , 

^ ' ■ '7 

velop an adequate student test score process. Eowevcr^^ as noted in Chapter 3^ 
a major f ;.>blem in the use of student achievement tests is that of finding 
tests who. 4 content is reasonably compatible with that oi ti>e school dis- 
trict^* ^Ui '^icuicsi an whose variety is adequate to qover various grade levels 
::.nd snSjec -.r. y^s indicatp.d in Chapter 5^ juriftdictiohs^uch as Seiiing^ 

n 

Oklahoma and Houston are attempcing to develop new tests covering more sub- 

^ _ ____ __ __ __ _._ ^ - 

jects and that a:.e more compatible with the schopl^s own carrdLculum. These 
tasks' can take v^i:: years. optioti that these di^^trl ts took, Vrowever, fs 

to begin with hatlbcally available tests, recognizing that these are not coin- 
pletely satisfactory fpr the long mh. This latter strateg5 may ^be pattlcu- 
lariy appropriate for school districts that are. ;^speclally concerned over the 
low perfonnernce of their studeats on tests a.ri have as a nmjor district bbjec-- 
tive the improvement of proficiency levels on such national or stat^ tests.' 

Non)-monetaiy perf orijQaace-by--ob jectives ^PBO) plan^ far.e fewer problems j ^ 

C _ , : _ : 

but adeqo^'e ^reparation r,ime is needed to gp'in an onflerstanding by 
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participants of how to select objectives and targets and to establish 
procedures for- evaluating progress towards the targets^ These tasks are often 
assumed to be easy and to take little . :ime. in general, this is, not sbi For 

meaningful objectives on topics such as student progress^ considerable time 

_ - - - - - - 

and effort may.be needed to establish reliable measurement procedures. Hyde 

Park (New York) took fivfe mouths to pilot test its PBO program with approxi-- 

matfely 25 percent of its teachers. * Santa Cla?: ^ \-. lifbrnia) used a one-year 

period for developing its, PBO prpgranis . ^ . 

To summarize, therte nre two major activities for which adequate time 

needs to be scheduled in developing teacher tn^centive plans r (i) time to 

develop the appropriate teacher evaluation procedures and time to ^(aia 

adequate teacher participations School districts, it seems clear, sljould aot 

rush into these plans without providing adequate fnie for^ and attend f to^ 

both activities^ ; 

Shbald- There ne an Appeal Process? * * ; 

The majority of the districts t.hat we ^^^nmined had .4 fbraal appeal 
process; a few diti not. The primary advantage of an appeal process is t^ac 
teachers who feel that they have been evaluated unfairly have the opportunity 
to air their concerns. The appeal process provides a safety valvr for 
teachers to vent 'their cbh«-erhs while af fbrciing them t'he Kb 
correct situations that may nbt have been hahdl?.d aciequAte^> - . -> lan's 
procedures. For the mbst part^ the scriool diswiricts with which we spoKe 
repbrted few appeals and fewer reversals of initial* decisionsi To some 
extent, this low level of reversals was due to the participecion of high-ieve5; 
administrators, such as superintendents^ in the original stward determinations 
(particuiariy ^n the smaller school districts) so that an appe^il goes to a 



Althoagh we i.oxj.td no dj;rect empiricai evidence ^on the vaiae of appeal 
processes, they are widely: used and appear desirable i 

Whii Er^condit 



Two related concerns appear most important here. The first is an ade- 
quate teacher-administrator relationship, arid the secbiiH the existence of an 
acceptable basic salary structure* 

The presence in a district of mutual trust between administrative per- 
sonnel and teachers is likely to greatly ease the problems in implementing an 
eLfective monetary incentive plan. The level of trust is likely to be 



affected by the type of teacher performance evaluation procedures u^d* If 
the procedures are based on super;?isory ratings , any pre-existing suspicion 
and misiruBt is likely to be exacerbated by a monetary incentive system linked 
zo the teacher evaluation process. In such cases, the use of a^mare objective 
approach, such as evaluation procedures based on student achievement test 
scores and other objective. data, is likely to be mdre palatable to mistrusting 
teachers. The reverse may be the case where there is a high level of trust; 
teachers may be more inclined Lo accept a supelvisdry rating procedure than 
test scores over which both parties have le.ss direct control. Ladue^ Missouri 
personnel ^jiaye reported that "success or failui^e of a Salary schedule based oh 
the effectiveness of teaching is dependent largely on the degree to v7hich 
there is mutual respect^ understandings and sympathetic professional relation- 
ships between evaluators and teachers ("the Ladue Evaluation Salary Program," 
October i980, p^iS — see t-adue case report )i 

A second ^patentiai prerequisite i- a reasonably satisfactory basic pry 
plani Some teachers and administrators currently believe that, merit pay will 
not be i^uccessf ul Until the teachers' present salaries reach an "adequate" 
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associations that as long as they believe teacher salaries are tod low across 
the board, they are less sympathetic to using funds for payments to only a 
segment of teachers. This problem is likely to increase in seriousness to the 
extent that the monetary incentive plan rewards only a small percentage of 
teachers. If, however ^ the plan permits a substantial proportion of teachers 
to receive awards^ if those awards are of a substantial amount, and if the 
teachers perceive that a primary problem is the willingness of the community 
to pay across-the-board increases, then the monetary incentive plans may be 
accepted as a partial substitute. 

Our sample is too snail to inaRe iiiiy definitive generalization regarding 
the importance of these two prerequisites, but they clearly are important 
considerations. 
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ettAPTER 7 

Perforisance-by-db jectives Plans for Teachers 

A potentially proinising approach for stimulating greater productivity and 
higher levels of employee performance, is the regular specification of 
performance targets. This process involves the formal identificati on of 



specific work objectives and targets for individual empldyees at the Ireginning 
of a given period and the subsequent comparison of actual, to targeted 
performance at the end of that period A performance targitihg program can 
be used as a means for evaluating employee performance or as a motivational 
tool in its own right i ~ - • 

Chapter 5 disct^sed procedures used in monetary incentive plans. In this 
chapter i we consider non-iiiDnetary plans — though much of what is discussed here 
also applies to Monetary versions^ ^ 

Uhlifce merit pay procedures, the motivational effectiveness of perfor- 
mance targeting plans does not depend on the provision of moaetary awards. 

^ _ •_ — 2 

instead, such plans a^e based oh the psychological theory of goal-settingi 

Goal-setting theory postulates that human actions are triggered by conscious 
intentions that are expressed as specific goals. A goal is what an individual 
(or a group) is consciously trying to achieve and constitutes *the most immedi- 
ate determinant of performance* According to this thebryi increased effort 
and better performance could be realized by providing teachers with clearer 
targets or by making their targets harder to achieve (more challenging). 



n We will use 'the terms "goals'* and "objectives'* interchangeably in 
-this chapter. - ' 

2. See^ for instance, Schwab and Cumiiiings (I97i3), Latham and Yuki 
(1975), and Locke et_£l^. (1970). 
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. Target-secclng procedures for teachers generally involvi^. the fbllbwlng 

major s:teps; , . 

\ 

1. At the .beginning of the performance period ^usually a semester or 
school year)^ individuai teachers meet with their supervisors to 
ag^ree on objectives for the coming months* 



2* Each objective is supposed to incorporate a specific target to be 
achieved, criteria and data collection procedures for assessing 
target achievement, a work plan for achieving the target, and an 
indication of the responsibilities of the supervisor (or 
administratidiil) in the target achievement, effort • 

3. During the performance peribd, teachers receive periodic feedback on 
their progress toward achieving the specified targets. 

4. At the end of the performance period^ the teacher and supervisor meet 
once again tj3 review the ^egree to which each target was actually 
achieved. This provides ah bppbrtuhity tb identify heeded' improve- 

* mehts and to begin to specify objectives and targets for the next 
performance peribd. ^ 

The application of targe t-s^et ting procedures tb iiiahagers is generally 

termed "mahagemeht by objectives" (MBO). The use of target-setting prbcedures 

for annual performance assessments of management and hbh-mahagemeht personnel 

__1 __ 

is frequently referred to as "appraisal by objectives" (ABO). 



The focus of this chapter is on the use of target-setting procedures for 
mptivat Xag^ teaching personnel, that is, for stimulating improved performance 
from teaching staff. We will use the term "performance by objectives" (PBO) 
to refer to the target-setting procedures reviewed here. 

The performance contracts for teachers popular in tl}e 195b's (ERS, 1974} 

represent ah early applicatibh of target-setting prdcediilres td the mdtivatidn 

-- __ __ _ _ _>_ i< 

of teachers*. ^Siich progralns determined a teacher s pay, in part, by the degree 

/ ^ . • 

to whic^ the teacher was able to imprbve student achievement levels (as 

measured by standardized test scbres) in areas such as mathematics and 

reading, "tmprovements" we^e measured, against pre^pecified targets. Such 

programs were tried by school districts in Keokuk^ IbWa; Hehbmiheei Michigan; 

and Cherry Qreek, Colorado,^ among others^ 6^ 



PBO has been widely discussed as an approach f or evaliiatihg :|:eacher • 
performance • For example, Manatt ^ei^^i^ (1976) and Redfern (1980) have both 
described teacher evaluation* prdcedures based on'FBOi . ^ 

There a^re a number of Important advantages to the use of PBO for 
motivating teachers : 

_ _ S. _ _ _ _ 
o the procedure Is relatively simple and straightforward In concept* 

o It is a highly participatory process giving teachers a major role in 
determining hoV their performance will be assessed--an_ element highly 
conducive to motivating people, especially prof essibhals *such as 
teachers. 

' d Jt providejs, in principle, a relatively objective basis for assessing 
teacher performance • 

b PBQ of f era a way to motivate employees without the use of monetary 

incentives. This can mean lower costs and aY9i^59?_9^_^^® emotibhal 
respbiiees often associated with linking pay to performance* 

b Since bbjectives and targets can be tailored for each teacher^ the, 
procedure provides an oppdrtunity to adjust ^or individual 
differences in class compositidn, sub ject difficulty , etci 

Dlsadvantagies of these programs include the foUowittg: 

o Differences in target-settihg practices among t^kchers can make it 

difficult to compare teachers br schools (e.g. targets may .differ 

among teachers as to their impbrtahce to the school district and ease 
of achievement). " ' ^ 

o The program may require cbhsiderable paperwork and supervisory time 
(an especially difficult prbblem fbr PBO programs invdlving teachers, 
whete the span of control is unusually large). ' _^ 

o it potentially requires that all staff learn a nlw and difficult 

skill - that of identifying and formula ting realistic, challenging, 
and measurable ^Job objectives an^ targets. 

d it may require extensive training of all staff. 

b The prpgram can degenerate over Vi petiod of time if not coupled wl^h 
periodic retraining, revision, rejuvenation, and — perhaps — rewards. 

In this chapter we focus dn fdur types of issues associated with PBO: 

design Issues^ implemehtatidn issues, dperation and maintenance ^sues^- and 

issues associated with the impacts and effectiveness of such plans. The 
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findings reported here have been drawn from the experiences of five sctiooi 
districts currently asing FBO^ as well as the literature concerning the ase of 
PBO approaches for teachers a^d other public and private sector etnplcryees. 
I (The five districts are the Brbwti Deer (Wisconsin) School District, the Hyde 
Park (New York) Central Schocil District^ the Ridgewood (New Jersey) Schbdi 
District^ Che Salt Lake City '(Utah) School District^ and the Santa, Clara 

___ ____ __•__ _ * 

(Calif brhia) Unified School District.) 

.' ♦ ^ . • 

Design Issues • ' - 

There has been considerable research by the private sector dhd the 
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academic community on the best way to design a performance targe tL^ system. 
Their findings suggest that the fqllewing features enhance the effectiveness 
of a PBO effort:^ . 



o ' Performance targets phouid'be clears specific, challenging yet real- 
istic, priority ranked, and — to the extent possible — quantif iablei 

^ o Employees should participate in the formulation of their goals and 
per.fdrmance targets. ^ ^ • 

.- b Overall objectives should be established at the top level of the 
organization and cbmmunicated to litie staff. 

_ _ _ - ^ .__ - ^- - - _ - _ - - ^ - - 

b The perfprmanc^ objectives, should be accbmpanied by written action 
plans indicating hbw the objectives will be accbmplished. 

b Emplbyees should be provided with frequent , relevant , timely, and 

cbdstructlve (hbh-^ritical) feedback bh their prbgress tbward meeting 
their objectives. ' ^ 

Other important design issues on which the research does not prbvide much 
guidance are: (1) the importance of focusing objectives and targets bh out- 
comes rather than processes^ (2) the net additional mbtivatibhal value bf 
linking target achievement to f in^nci^l^rewards, and "(3) the heed for central 
coordina^on and oversight of employee efforts to formulate perfbrmahce' 



Im These results are reviewed in greater detail in Greiner et al. 
(1981), Chapter 8, and the ^references * cited therein. ' 

m 



. targets Private sector and academic research findings on^these issues tend 



to be either ambiguous or non-existent i There is some limited inforination on 
these* concerns fron? a recent study of nE our public sector PBO prdgrams (riatry 

*._ •_. " < , - - _ • . / : . J 

-et^ i 3X^ , 19^2) ♦ (These pcogramis invdlved several fcinds of municipal empldyees 



but no teaching personnel.) It was fduh<i^ for example^ that: 



There is evidence that perfdmance targets that fdcus specifically 
and explicitly on prdductivity-^.g. ^ butcbme measured employing 
efficiency or ef-fectivehess targets-- *ave the , greatest likelihood of 
fostering prbduc^tivity improvements. y' 

j " . - - .-^^ " 

Prcfcess tar'gets ibf the form "identify and impieifent at least one 
prfeject relatedjtb cost saVingis or service improvement over the next 
12 months" have^'often b^en effective in -stimulating employees .to 
undertake special projects that improve productivity^ f 

PBO programs where target-setting was highly decentralized and lack- 
ing iq central staff oversight tended to exhibit large variations in 
the quality and difficulty of the targets formulated by employees. 
Withou^. such centt'^l coordination, target quality tended to decline 
with time. ' ' ^ , 



These findings, while' tentative and based* on public sector PBO programs 
involving non-tea^ters, are potentially relevant to the -design of PBO ap- 

proac^hes'f or teaching personnels .Nevertheless, tJhe precise extent to which 

* * 

-- -- _* ^ __- ^ _ _- i - _ - - ^ - 

findings^ on PBO programs fdr ndn-teaching 'personnel In the public and. private 

sectors ;are applicable td public school teaahers is not clear. While many of 
those findings are likely tb ^pply^ there are pbtehtiall^ impbrtaht differen- 
ces between PBO programs fbr teachers and other PBO programs. For example^ 

the complexity of perfbrmahce criteria associated with teaching is getierally 

/ . ■ » 

_/ ____ __ __ 

greater than fbr the work of privat;e sector employees; heavier emphasis is 

given to process measures in assessing teaching performance; there are cor^- 
straints imposed on teacher PBO plans by l^s^ rules, and civil service regu- 
lations; and there is a tradition of strong' emphasis on sei\lority and training 

rather than merit in conjunction with teacher compensation planss* An example 

i - ^ 

of a potentially important difference between teachers and other ^public sector 
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employees, insofar as PBO programs are concerned ^ i^s the reiativeiy large span 
of control usually associated wi-th the supervision of teaching persormei (Gray 
and^Burns^ 197.9). "Hxis means that a single school principal may have to 
evaluate a large number of teaching staff; a situation that can lead to 



Ineffective evaluations and reduce the rndtivationul-ef f ectiveriess of a PBO 
program. ; » 

Nevertheless, th^ five case studies and the results we have found from 

research on other public ^ec'tdr PBO progtams illuminate a. number of issues 

, „ ' , ' ' ' . t . 

regarding the design of ' FBO programs for teachers. The fdlldwihg issues 

' ^ ' >^ 

appear to be especially significant. ' ' 

^4 Program Coverage . * i 

Each of the five PBO programs initially covered all teaching personnel./ 

._ V _ !_ ___ 

However^ in two cases coverage was curtailed after a few years. In Brown 

Deer^ teachefs were given ^the: option of selecting one of several performance 

_^ 'i] _J __ 

appraisal procedures whose emphasis on goal-getting varied considerably. In 

Hyde Park^ the PBO program was restricted ,^o non-»tenured staff and teachers 

with performance problems^ Such curtailntlnts in coverage make it' difficult ^to 

view these target-setting procedures as / real PBO' efforts at the current 

time. To be effective as a general motivator of teaching staff, it would 

' appear that PBO programs should cover all '(or at least most) ^teaching 

personnel. 

The' Structure and Source of Teacher Objectives . 
- All five P^O programs initially used a hierarchical objective structure, 
. with district-level goals (specified by the board of education ahd/dr the 

% » _____ _ ; 

superihtehdeht ) at the tbp^ "building" bbjectives "for individual schools 

a •- _ _ _ _ _ _ ; _ _ - 

(formulated by the school principal J at the next levels and the objectives for' 
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individual teachers' at the lowest ieveii Buiidingt and 'district bbjecclves • 

wi^re expected to be integrated with and proviji€^uidance to the Sormuiationi of 

/ - 

objectives by teachers i ' 7 . . • * ' < 

\ * ^ 

Job cliar'acteristics and behaviors incruded on the rating forms ^sed to 

appraise teacher performance also frequently served as a source of objectives 

(see the Hyde Park, Ridgewbbd^ and Brown Deer case reports), as have the 

^ » ■ 

minimum basic skill levels that some school systems specify for their students 

(Santa Clara and Salt bake City case reports). Ah innovative source of 

teacher objefctives in the Salt Lake City schoosl district has been the "School ' 

, Community Councils''- associated V7ith each of the district 's ' schools (see the 

Salt Lake City case report )i ^ v 

The Focus of Te acher Object i ve s^ 

Few of the objectives established by teachers in the five PBO prograc^ 

examined here' appear to have focused ots educationai outcomes or efficiency. 

Xh those cases where objectives regarding student achievement were reported ^ 

(see for instance the Salt Lake City, Ridgewood, and Santa Clara case 

reports)^ they often tended to be relatively general (e^g. "improve the 

teaching of spelling") or to focxas on the basic skill objectives specified by 

_/ __ _ _ 

the school board Je*g. "teach long division concepts using the approved 

curriculum in such a way that 90 per<;ent of the students will pass an exain on 

long division") i There were, hdv/ever^ some exceptions* For instance, in Salt 

Lake Cit^ many teachers established objectives for achieving cost savings (in 

response to ,a similar district-wide objective). 

Most of the perfonaance targets reported by the five school districts 

focused on teaching processes, the execution' of special projects, and/br prb-» 

fessional development. All of 'these tenc^to focus on the ^eans. to an end * 

rather than the outcomes of interest* Teacher performance appraisal 



procedares using target-setting straddle the line between competency-based 
(iiei process-orienteS) and outcome-based evaiuatibn philosophies (^Daxrlihg- 
Hammon^ ^.ai^, i9fi3). Thus, depending on how the: objectives are defiheds PBO 
procedures 'can focus on either outcdmefor process msa; :Tres of performances 
Unfortunately, V the precise relatidnship between achieving a procesn-oriented 
objective and improving educational outcomes has not usually been systematic- 
all5r established.' The same can probably be said for many perfbrmarice targets * 
focusing oh personal development or the cdmpletidh of special projects. Such 
efforts generally have tended to be largely remedial or to focus _ori curriculum 

_ ^ _ _r " _t "T 

development and other changes in the teaching process ; their relationship* to 
oatcomes remains problematic i ' ' 

An especially important issue for . teacher PBO programs is the ^egre^ to 
'which a teacher's performance objectives focus oh the remediation of deficient 
'c'i'es , "^R.^pppdse to the encouragemept of excellences Many pf the teacher PBO 
prdgrams examined placed their greatest stress on using the targets to ^^^^ 
identify and alleviate areas of poor perfonnance (see, for instance, the Hyde 
Park, Santa Clara, and Salt Lake City case reports). While such a focus was 
apparently effective in eliminating "ptdbl^m" teachers, by itself it is likely 

. _ •; \ __ . 

to do little tcr encourage dvetall excellen^. 



•r^'w * Sel^^ctibn of Specific Pgrfbrmahce TargetS r 

While goals and objectives identify the general areas .df concern (e.g. : 

student achievement^ curriculum development^ or ah aspect pf teacher be- 

^ I 

havior), it is the performance target that spells but the specific actidnor 

change being sought * Vbr instance, if the bbjective is to improve reading 
skills, the :corresponding perf ormahcet target might be "improve reading levels 
by 1.0 years over the ^xt six 'months as measured by a specific standardized 



testi" 



Many of' the objectives we- examined did hot actually spe^cify any perfbr-*"^ 

• ... . " 1 ' ' .. ? 

mahce targe ts » Some did not even. define the peribd to which rhe performance 

'- ' * \ ' 

target applied', although most of phe. targets examined used the school yeir or 

; •/ 'i\ , J. • ; v_ 

the time until the last scheduled performauce evaluation (usually in April) as 

the (implicit) perfbrmahce period of interests 

Another issue in foniulatihg meaningful performance targets is how tio 

J' ^ 

djeterniine the specific level of performance to be sought. For instance, if 
the objective i^ an improvement in reading scbfcess how does one determine, 
whether to seek an improvement of 1 or years in such scores? A number ^f 

^ . '_ . ^ ._ . 

sources might be used to specify such, -targets : prior levels of achievement; 
• historical rates' of improvement; state' or prof essiohal ^SXandards (especially^ 

for schools or classes that are- below standard); district goals^ or policies; 

* ' _ _ _ f _ _ _ . 

or perhaps the .actual perfornsance of othe.r classes or schools in the district^ 

the state I or elsewhere in the nation. . As noted pre^^iousiy ,.:.twp school 

districts (Salt Lake City and Santa Clara) used as performance targets the 

basic skill objectives that the school board had adopted for each grade. 

■_ 

'the specification of the actual perfbrmahce targets is an especially 

important cone^^rn for PBO programs designed to stimulaf* outstanding perfor- 

- «' _ • _ .0. _ _ _ 

mancei: If the PBO plan is to serve as a stimulus for excellence , it>may be 
•inadvisable (depending upon ^ow targe t achievement is^ measured ) ^^^^^ ' n rf ^ j nu m 

perfbrmajice s^and^rds- as the targets.. Meeting a minimum perfbrmahce stariiK^;^ 

_ _ - -iU'^' 

cannot, in itseif, be consider^^^ a. mark of excellence. For this reason, usl^g 

minimum basic skill objectives as^erf oroance targets may nat be the best way 

to stimulate excellence. The question of jnse what kind and what level bf 

peffbnpance^ target's do stimulate excellence In teaching is a difficult issue 

but one that must be resblved by any school district attempting to use PBO as 

■ ■ - _ _ " * • ' ' ' 

a general mbtiv^tiohal tbbl« ^ . _ 




•A related issue is how. to adjust for differences between teachers with; 
regardr^to the subject taoght, student mix, etc^ If such differences are :not 

_i :^ ^_ . __ . . __. . • _ 

considered and informatiori on the achievement of perfoTiaance targets is us'ed • 
for* purposes important to teadhers (e.g. as a substailtial part df^annual per- 
foirmance Appraisals or as a major basJLs for recbgnitibii and auards), then 

V ____ ' _ _; 

charges of ' inequities aiid u^airness are lifcely to occur. Some teachers will; 
feel that other teachers havq had easier ox less 'importaht (substantive) 
targets. The five teacher P50 programs examined here used ho formal^ syste^ 

matic approach to adjust for such differences. Ih.sbme cases (e.g. Santa 

^ • _ X I-. ' _ _ __ 

Clara thct i^sue was' partly addressed by stressiijg to teachers the importance 

- -_ ^ _ ^: 

of tailoring their efforts to the needs and abilities of each individoaiy 

^tudeht. Teachers were rated specif icaiiy on their effectiveness and 

conscientiousness in carrying put this requirement. 4 

But in geiteral, it was up to the t^achers^ ot supervisors to make any ^ - 

' - # 

adjustments, eigi by adjusting the actual performance targets or by subjec- 

_ _ ^v- _ _ _ ,. _ ' r ' _ ^. - ■ • 

tively allowing fo^r such facttrs when evaluating target achieveiSiedt . An 
approach sometimes used elsei^here in tliig. public sector — central njonitoring ^hd 

, ' ^ ". ' , to 

review of all targets to contrdi target comparability— <Jas not used in the 

- - - - . - - _ i - - - - - ' - I 

school dl^Criccs exanHiied here. (Howev^ri Brown Deer did istrive Cd enhance ^ 

^. - - - - - f ,__Q_'___ __ _ 

comparability between schools by holding joint discussions of the various 



performance ratings with all three schpal principals present.) 

- ' ^ - _ ~^ * 

^Although thii issue of t-arget comparability has been a cfentral cbhcern for 

many PSO programs ^ it does not appear to have been a major issue in any bj the 

schbbJL^ districts examined here. This may, o£ course,, reflect the absence b£ 

' ' 

rew^j'ds or sanctions in connection vd:th target achievements Concern over 

^ - ----- 

targets^^mparability and inequities would probably have been more severe had 

any. of these school districts linked target achievement ti^financial rewards i 




Hand Iri'Tlarid with* the problem of specifying objectives and performance 
targets for teachers is the issue of how to measure target achieveiaenti One' 
of the assumptidns-~ahd|.. perhaps^ biases—associated with the use of . PBO is " 
the measiirability of major objectives. Very few of the perfbrmarice targets 

_ _ ^ - - - _ _ 

for teachers we reviewed abtually specified the criteria and data sources to 

be used for measjt»ring target achievement. Those that did usually used prbject 

products, due dates^ or rest scores for making such assessments. 5ut Xn most 

cases, it was up to the teacher or the evaluator to muster whatever evidence 

was available to'determxne whether the target had been achieved. 

in addition (qtr ^perhaps, as a result), in many instances target 

, . » , ■ 

achleveinent was assessed only subjectively, using whatever information the 

supervisor and the teachef could put together. The subjectivity inherent in 

such ah assessment process can obviate one of the important potential 

advantages of a PBO system^the ability to provide a relatively objective 

assessment of teacher i^rfbrmahce. The fa^iure-^-br inability — bf inany 

teachers to specify appropriate measureS^of ta^-get achievement can thus lead 

to a PBO plan thac is jud| as subjective as many bther supervisbry fating 

" procedures.^ This appears; to have occurred In cbhhectibh with many bf the 
teacher PBO prograqis we .^amined. ^ 

The subjectivity noted above was often compounded by the v/ay in which the , 

" assessments of target achievement were used in determining a teacher's o^rerall 
perf drtnance. In most ca&*€(s, target achievement was only one of many inputs 
considered in prdducing ag essentially subject tye- overall perfoignance rating. 

The precise manner in which target achievement results were to be incorporated 

^ __ _ » _ _ _^ . 

into the overall perfbrmahce rat,fhg was rarely spelled outi The essential 

subjectivity of the resulting ratixi^, and its Ibdse relation tcT'Carget^ achieve- 

'm^ni ^^Toiiph^ diluted the mbtivatibhal impact of the- PBO plansi 
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The Mech aalcs— o f the Obj ective- and Target-Settitig Efforts 
An Impdrtant^ issue here is the number of targets that a teacher shdiild 
set. Several of the school districts examined (e»g»| Santa Clara and Hyde 
Park) initially had teachers formulate numerous perfbrmance objectives and 
targets. Udweveri hone of the five programs are now using more than about six 

objectives per teacher, and there appears Co be a general [perception that the 

__ 1 _ _■ __ __ _/ 

number of targets for a given teacher should be kept to between one and six. 

Another qaestipn is what should be the respective roles of the teacher 

and the teacher^s supervisor in setting the performance targets. In two of 

the PBO programs examined here (Santa Clara and Salt Lake City), teachers tddk 

the initiative in preparing a first draft of their perfdrmance targets for the 

coming year. In the remaining three cases, the perfdrmance targets were 

drafted jdintly by the teacher and the teacher's supervlsdr. These prdcedures 

have the advantage of making it more likely that the teacher will accept and 

seridusly strive td achieve the targets that are finally agreed dh> A jdiht 

^|^rget*settihg procedure helps ensure that both the teacher and the teacher's 

supervisdr iihderstahd the teacher's objectives for the coming year and agree 

on their importance (McGreal^ 1983). 

c 

On the other hand^ the specification of tekcher perf oirmance targets was 
not always entirely democratic. When teachers exhibited performance defici- 
encies, principals often took -the initiative in specifying remedial targets. 
In addition district and school building objectives imposed farther limits on 
teacher flexibility in setting performance targets. Ijx assessing the problems 



they had encountered in connection with their first PBO effort, Hyde Park'school 

----- - - \ ■ - - - 

administrators stressed the importance of having supervisurs take some initia- 



tive in setting targets, especially in cjdnnection with targets addressing 

X • " 
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district-wide priorities (Gray and Barns^ 1979) i this participation by 
supervisors can ensure that the targets of Individual teachers are of 
Importance and are compatible with district priorities. 

The Relationship Betveen PBO and the Teacher Evaluation Process > 
The relationship between a school district ^s PBO program and the 
procedures used for annually appraising teacher performance can be quite 
complex. Of the five teacher PBO -systems examined in the course of this ^ 
study, only one (that in Salt bake City) is currently used as the sole 
procedure for appraising teacher perfprmance. ttyde Park aft^' Btuswn Deer 

initially used their PBO procedures as the primary mechanism for appraising 

- \ W 

teaching personnel. However, both de-emphasized the rolej of PBO in 
performance appraisal after several years and began to u^ goal-setting merely 
as an adjunct to other perfbrmance appraisal techniques (see the Hyde Park and 
Brown Deer case reports). In the case of the other two school districts 
(Ridgewodd and Santa Clara), PBO has always played a relatively limited role 
in appraising overall teacher perfbrmance. 

The Brown Deer School District moved tbv/ard other perfbrmance appraisal 
methods in part to help alleviate a growing threat that teachers perceived in- 
connection with the use of PBO to assess their performance. Hyde Parkas de- 
emphasis of PBO came in the wake of its inability to maintain the effective-* 
ness of the PBO system over a period of several years (a similar decline^ also 
potentially traceable to inadequate maintenance of the plan, has been reported 
by the Santa Clara School District) i 

These results suggest that if PBO is being relied on for performance 
appraisals. Inadequate maintenance of the PBO program may damage its 

■i 

effectiveness as a source of appraisal Information, leading to pressure for 
de-emphaslzlng or dropping the program. On the other hand, subordination of 

110 



PBD procedures and results to other perfbrmahce appraisal techniques may 
seriously dilute the objectivity and mbtivatibdal effectiveness of the PBO 
program, as discussed before, and may , pr event one from realizing the * 
advantages of PBO as a motivational tool. 

These results suggest that school administrators contemplating the use of 
?B& systems give careful attention to the relationship between PBO and the 
procedures to be used for rppraising overall teacher 'performance. We suggest 
that if the PBO procedure is not made, a significant part of the teacher* 
evaluation process^ it will haye little meaning to teachers and little 
motivational value. . « * 

the Relationship Between Target Achievement, Rewards, and Penalties * 
As noted previously, the question of whether or not to link target 

achievement to rewards and penalties has not yet been clearly resolved in the 

_\ ■■• '_ ' ■ _ _ 

research literature. Nevertheless^ a decision on this Is&ue may be quite 

important to the motivational effectiveness of a PBO program. 

School administrators in Hyde Park and Brown Deer both f^l^ that the lack 
of tangible awards for target achievement had contribated tp tlie decline of 
their PBO efforts. Indeed, the presence of some form, of sanctions or awards 
may be necessary if teachers are to take the PBO process^ seriously over the 
long run. / These need not be monetary* For instance, strong support of the 
PBO effort from top-level management may in effect create a psychic reward for 
taking the program seriously. By limitiiig target-setting to non-tenured 
teachers and teachers whose performance was poor, the Hyde Park School Dis- 
trict in effect focused its PBO effort on those situations for which strongs 
sanctions (the withholding of tenure^ the withholding of step increases, or 
even termination) were available to ensure that the plan was taken seriously 
by those involved* Several other school ^ilrtri^ts ^ Salt Lake Ci^TT"^ — : — - 



Clara^ etc.) noted the prominent arid effective roie played by PBO in identify- 
ing and remediating poor performance — ^anbtHer case in which PBO seems to have 
Seen most effective when target achievement levels were associated with clear 
sanctions or rewards. 

On the other hand, there appears to have been an asymmetry between the 
rewards for high target achievement and the penalties for not meeting one's 
targets in the teacher PBO programs we examined^ While most school districts 
linked low target achievement to the initiation of remedial actions and, if 
necessary^ dismissal, high levels of accomplistanent with respect to the year's 
targets were rarely given much recognition beyond an occasional pat on the. 
back and 'a notation in the employee's personnel folders By failing to provide 
adequate recdgnitibh for outstanding target achievement while emphasizing the 
negative implications of not achieving targets, school districts may be 
failing* to take full advantage of PBO as a positive motivational tool for 
improving employee performance. 

Implementation Issue s^ _ 

Although the evidence is limited, it appears 'that the implementation of 
PBO programs for teachers is not greatly affected by pre^bhditions such as 
pay levels or prior experience/witl^ target-setting. The presence bf uniphiza- 
tibh and collective bargaining agteements also did not appear tb have affected 
the feasibility of the PBO efforts in the five school districts. The absence 
of a direct linkage tb teacher compensation is probably a major reasbn why* 
these problems have not tended to arise in the PBO piaas we examined. 

A few implemehtatibn obstacles were reported', however^ Hyde Park 
reported that cbntractual constraints on the teacher evaluation process and 
the large span of control characteristic bf their schools made it more diffi- 
cult tb initiate the PBO approach and may have affected the effectiveness and 



staying power of the program chat finally einerged (Gray and Burns, 1979) • Two 

school districts reported special problems in getting secondary school adtnlii^ 

istratbrs to utilize the PBO approach effectively. 

Most of trhe five scEbbl districts made ah effort to involve' teaching 

staff in the deveibpmeht ahd/br maintenance of their programs. Mbrebver^ the 

PBO procedures were asoaiiy incorporated ^into the provisions of the teacher 

contract. Thus, 'although the employee associations may not have had a formal 

role in the actual development of the PBO system, the need to obtain their 

con>tractual agreement to the provisions of the plan probably ensured that 

teacher assdciation concerns with regard ^to the PBO effort were taken into 

cbhsideratibh (this may, of course, have tended to dilute the plan and limit 

the way it was used). v 

Adequate training in writing Joals and using t her PBO process has bee^ |^ 
■ . ' . ; ' ' : _ _ _ . _ - . V 

viewed as essential by most experts bh the topic bf target-^settihg (see, fbr 

instance^ the results repbrted in Greiher et al. , 1981|^ p. 162). Two 6f the 
five school districts repbrted extensive effbrts tb train their' staff in writ- 
ing goals and objectives and utilizing the target-j-set^ihgssystem (see the , Hyde . 
Park and Santa Giara case reports). Whether that was ehbugh iS| howeveri un- 
certain^ since the PBO programs in both' sites apparently encountered prbblems 
in execution after a few years. Schooi districts Implemeh ting a PBD process 
should provide for j extensive staff training in the development of objectives^ 

and specific targets and, for supervisors, in conducting teacher evaluations 

c • _ ■ ■ I , 

_ _ _ _ _ '_ si 

based on target achievement, this training need& to be provided to all new 
employees. In addition, periodic refresher training is also tieeded. ^ 



Operation^ arid Maintenaiice Issues- 

TW6 impdrtarit cbhcerns in operating PBO problems are: (i) that it may be 
necessary for a district to laurich a/majbr effort to collect new or additional 
data on teacher performance^ and (2) that PBO programs can lead to 
^considerable extra paperwork for teaching and administrative staff if efforts 
are not made to keep such paperwork within bounds. 

The five px-dgrams we examined, however, generally required no special 
bperatibrial support — data processing facilities, specialized staffs etc. A 
potentially cbritrdversial issue associated x^th many target-setting ef fbrts— 
the conditions under which employees are allowed to alter or adjust their 
performance targets, during the perfdrmance period — did not^app^ar to be a 
source o£ cbhtehtioh in the five schddl districts examined^ All five 
permitted teachers to revise their perfbrmarice targets at any time with the 
concurrence of their supervisor. Only targets addressing poor performance 
and/or a termination decision could not be modified once the performance 
period had begun. 

the lack of coiproversy cbhcerhihg the fairness bf the targets or revi- 
sion df the targets 'may reflect the absehcer^of any direct linkage between 
target achievement and financiai-^r other—rewards. Such a linkage tends to 
focus atteritibri upon the equitabiiity of the targets^ making adjustmedts df 
the targets to reflect changing external cbhdit ions much mbre urgent arid 
cbhtrbversial. v 

The prbvikibri bf frequerit, timely feedback on target achievement has be^ri 
cited by many as ari impbrtarit factor in the effectiveness of a PBO prbgrOTi V 
(see, for instance, Greirier et al . , 1981, pp. 158-160). The f.requency of such, 
feedback in the five school districts we exainined ranged from one or twq^ times 
per year to weekly or even daily. (The instances of more frequent feedback ^ ; 



generally- involved PBO programs that focused on only a few teachers-^-e ;g« ndn- 
. ^ . > > ■ ■ ■ ■ . . . * . 

' tenare^ staff and persons needFng remediatidn;) In several cases the feedback 

frequency reportedly declined over time, Most^ of the five PBO ptograms either 

did not specify a minimun frequency for providing feedback br^requii^ed only 

one or two interim reports on target achieveipent • Given the ^motivational 

_____ --^ 

imp9rtance. of regular feedback on target achievement and the wide range of 
feedback frequencies reported by the five sites ^ it appears that school 
districts contemplating the use of PBO as a mbtiVationai tool should probably 
require supervisors to provide feedback often enough to ensure th^t the 
motivational potential o£ the program is not compromised^ 

Three of the five school districts ekamined — Brown Deer,- ^de Park, and 
Santa eiara — reportedly experienced a decline in their program^ after several 
years. Among the problens reported were a decline in the quality of the 
V objectives specified, declining awareness and understanding of the PBO 
process, less cdnscientioiisness by supervisors in observing teachers and 
carrying out the evaluation process, anH growing jeinfildyee j>erceptidh6 that the 

PBO prdgram cdnstituted a threat. As ndted by Santa cl^ra Schddl Distrltrt 

^ • . _ _ 

officials, these symptdms pdint to the* need for periodically reviewing aiid re- 
vitalizing the PBO prdcess if it is to survive and be effective. (In fact, it 
did ndt, survive as originally designed in either Hyde Park or Brown Deer.^ 
Such revitaXizatidn can take th# fdxtii of cbhtihxiii^ to train 'supervisory and 
ndh-supervisdry personnel in the techniques and applications of PBO, frequent 
rotation of Supervisory staff to biding In new perspectives for evaluating 
teachers^ ahd--perhaps--— the dhtrbductlon of some frrm of recognition for 
butstandihg target achievement. 
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The Effectiveness of PBQ Prog^ami^ for Teachers 

Private sector experiences and laboratory research on the use of . 

performance targeting programs suggest that^ wl|eh properly designed and 

applied^ performance targeting can have a positive effect oh productivity 

(Greiner , 1981, ppi 148-i5i)i There have^ however, been few effprts to 

assess the effectiveness of perforctance targeting programs for public^ sector 

employees (Greiner ^^X^ , 1981, ppi 143-145) and virtually none" in cohhectioh 

with PBO jptbgrams for teaching^ persbnnel. In dxie of tta few recent studies of 

the impact of performance targeting on public sector (l^t non-*teaching) 

employees (Hatry et al > 1982), it was found that: 

6 The target-setting process itself appeared to be responsible for a 
number df modest thought scattered productivity imprdvenientsi 

b Performance targeting efforts did hot produce any significant amount 
of job dissatisfaction* 

o The linkage of target achievement to mc etary rewards did not appear 
to produce any imprdvemehts in performauce that would not have been 
obtained had no siich linkage been used. ^ Indeed ^ the ihtroductioh of 
monetary awards generated considerable dlssatisf action with the 
entire PBO effort, 

o The targets with the greatest likelihood of fostering prpductivity 
improvements were those focusing specifically and explicitly on 
productivity^ e»g»> outcome measures employing efficiency or effec- 
tiveness targets^ ProcesB-oriented targets reqixirihg employees to 

? regularly undertake special productivity improvement projects were 
also found to be. relatively effective in stlmalating improved 
performances 

Thus, the experiences of other public sector empio^^ees seem to indicate 
that if properly designed, PBO programs can stimulate modest improvements in 
perfbnnahce without having to be linked to monetary, rewards i furthermore, 
such programs apix^ar to fcave few iiegatiye effects. An especially attractive 

Its frequent/ success in motivating management and profes- 
a fact ttet makes its application to teaching personnel 



aspect of PBO is 
sional personnel^ 
especially promts 
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Orie miifit^ nevertheless^ be caucioas in extPapaiating the resales fr din' the 
private and public sectors to PfiO programs for teachers. As noted previously^ 
the* latter programs may involve some significant differences from those fox 
non-teacfeing staff — e-^g., the need to deal with the large span of cdntrbl and 
the complexity and subtlety ot the butcdmes associated with the teaching 
process. Ilius, i&f ormatidn is needed on schddl district experiences with* PBO 
prdgrams ^nd their effectiveness for improving the perf brmahce of teaching 
personnel. Unfortunately^ we have found no systematic evaluations of actual 
schbbl district PBO experiences that shed' sigiiif icant light o£ their impacts^ - 

Three major reasons why school districts introduce PBG programs eme,rge 
from the five programs examined: (i) increased accountability, (2) improve- 
meht of the performance appraisal process, and (3) satisfaction of state 
requirements for evaluating certificated personneii it is significant that 
none of' the PBO programs examined were specifically designed and introduced td 
stimulate better performance by teachers. Indeed, tl^e inattention by these 
school districts to many of the design and implementatidn issues delcribed 
above suggests that the mdtivatidnal potential df such prdgrams has generally 
not been tapped by the schdol districts. While PBO programs can potentially 
stimulate excellence cn'^the part df teachers ^ they must be designed^ imple- 
mented, and maintaiaed with that in mind. { 
It is thirefdre perhaps ndt surprising that the five PBO prbgraxiis we ' 
examined appeared to have had little impact^ positive bt 'negative, on teacher 
perfbrmahce and attitudes. Only in Salt Lake City, where PBO represents the 
only perfbrmahce appraisal procedure^ did administratbrs credit the plan witH 
cbhtributihg to improved academic achievements by students. However, no sub- 
stantial independent evidence of this is available. When focused narrowly on 
non-tenured staff and teachers with perf ormance problems (and'*" therefore « 
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'potenciaiiy facing dismissal)^ perfonsance carget-ing efforts have been 
reported to be quite effective in shaping and' imprdvi,ng teaching practices ai|d 
procedures; (See also Iwahicki, 1981, p. 203*)* But for the large majority of 
the teaching staff, no specific changes due to PBO were rep^jzted. Only one of 

the five districts (Salt Lake City) credited PBO with affecting teacher teteh- 

> - * 

t.^on or recruitment; in that case the PBO ef.fort' was cited as facilitating the 

/ . : 

elimination of poor and unsatisfactory teachers and the retention of more 
effective staffs * 
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Labor-management relations were also generally unaf f ectj£d_hy_the^^ 
intrdductidn of PBO programs for teachers. Indeed, the specifications of the 
plans were usually incbrpbrated intb the teachers' contract. 

The PBO prograois -examined here appear to h^ve had little impact, one way 

or the other, on teacher morale and' jbb satisfaction* The Salt Lake City 

School District^ which appear^ to emphasize PBO more than the' other four 

school districts^ reported some (miiibr) negative feelings towards t^he plan on 

■% _ _ . . - _ _ _ ■ 

the part of teaching staff. And teachers in BrbWh Deer repbt^edly began to 

view the PBO effort as somewhat threatening after several years. But for the 
most part^ teachers appear, to have accepted the PBO process. (Note that, 
acceptance by primary school personnel was in some cases greater than that by 
secondary school staff.) ^e presence of state laws in California and N^w 
Jersey addressing the need for careful performance appraisal procedur^a^s and, 
to some extent, encouraging the establishment of objectives appears to have 
contributed to staff — and union — ^acceptance of the plans. The fairness of the 
PBO procedures has not generally been ^tv, ijs]^u<4 in the school districts exam- 
ined, possibly, because of their highly ^L^ mited use in the teacher evaluation 
process (except in Salt Lake City)^4 / ' 



%fi general, the dut'-df -pocket ^dsts assdciated with a PBO ef fort for 
. teachers, appear to have been relatively low, although the school distric^ts 

_ _ _ -_ - _ ■ y 

examined have hot made a detailed accounting of the expenditures associated 

with their programs. To the extent that PBO is used as the primary 

performance appraisal procedure (rather than as a separate motivational tool 

or as an adjunct to other performance appraisal procedures), the x:ost of such 

a program r^epresents an expense that would probably h€ve to be borne by the 

school district anyway. / 

Some school districts reported^ that t^eir PBO prdgratns required special 

ej;periditures for cdnsultants, trainers, ahdX^r extra recdrd-keeping, but no 

estimates of the magnitude of these costs were available.' The latgest cost 
\ . . _ _ __ __ ^ _ . ' 

associated with a PBO effort is likely to be the t;ime needed for fdrmulating 

and negotiating objectives and ^ subsequently meeting to assess' target achieve- 

* . ^ ^ _ _ _ _ 

ment. For instance^ the comprehensive PBO effort employed in Salt Lake City 

reportedly is quite time cbhsumihg^ ihvblvii%« the cbbperatibh and cbmmitmeht 

__of many people* 

The school districts exaid^^ noted several other advantages and dis- 

» 

advantages in connection with their PBO programs. For instapce,« one district 

_ / 

suggested that the target-setting process cohtributes to iik>rbved continui4:y 

in district activities and emphases from one year to the next. On the other 

hand, it was suggested by others that the district and boiiding objectives isay 

- : , 

Increase conformism and reduce flexibility and initiative on the pai:t of 
individual teachers. 

In summary, while PBO prdgrams fdr teachers can, potentially stimulate 
improved "performs (if thfe results fdr PBO prdgrams for non-teaching staff 

ckh be extrapolated tb teachers)^ tha$ potential has ptdbably not been fully 

_ _ ■_ . . _ _ _ _ _ _ K - — 

jixploited by the schbbl districts ' examined here. While these f ivevprdgrams 
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offered iittie evidence*' of significant positive (or negative) impacts on 

-^-■^^--^ - - -- -' - - . ' ^ "- 

/ edacacidimi oatcomes and teacher performance, these results should be viewed 



with -caati^ru the five PBO programs i^ere app^ently not designed or used 
ispecificaliy \o motivate teachers » Thus^ the nrbgrams frequently cbhtaihed 

- -\ ~ - - - • ----- 

design and implemehtatibh features that failed tb take advantage bf the Jull 
Hiotivatipnal pbtehtial of PBO. Fbr example: *^ ;^ 



b ^ In some cases^«the program was applied to only a sm^ii portion of the 
teaching staff* ^ y ^ ' ^ 

6 the performance objectives and targets that were used tended to focus 
only on processes rather than outcomes. 

o .feedback on target achievement tended to be infrequent i 



o Performance criteria and targets were ^ftett highly subjective; 
specific, quantitative performance targets were rare. 



o The linkage between the PBO process and a teacher's overall annual 
performance appraisal tended to be cdmpXex and to de-emphasize the 
importance of target achievement In favor bf subjective ratings bf 
humeroUs other characteristics^. 

d Little recdgtlitlbn was provided td teachers who perfdrmed well with 
respect tb thei*r targets, although inahy, school administrators 
believed that such recognition coi|^d have a _ positive impact oh 
-^btivatibh. In(^ed| greater attentiohwas focused bh sanctions for 
pbbr perfbrMhce than oh recbgnitibh of ^bod perkbrmahce. 

o Only spbradic attehtibn was paid to maihtalhihg and revitalizing the 
PBO effort after the first several years. Remedies focused bh 
altering the perfori^hce appraisal procedure rather than examining 
the factors assbclatetl with Imprbviag emplbyee motivatibh and 
perfbmahce. Strategies for helping to refocu^ employee attehtibq 
and energy upon the PBO effort (e.g. the introduction of sanctions 
and/or rewards for giving serious attehtibn to the PBO .process]^ were 
not generally considered. 

o Where quantitative performance targets had been established , they 
generally represented mininal ^ perf drii£^nce stanjdards radher than a 
goal of excellence. 

.. \ 

Without attention td the fdregping issues, the motivatidnal effectiveness 



o£ the p'BO prdgrams examined here is likely td have been compromised. . thus, 
while the experienced to date with PBO prdgrams for teachers illuminate the 
issues. Associated with their Use as a stimulus *f or imprdved perfdrmanGe, they 



. . - ... . ^ I- ^ . • 

do not appear to provide a basis for assessing the actual}*pdtential effective- 
ness of such programs for mbtiv%iing teachers/ While a nuinber of pros :ihd 
cbiis can be identified in connection with the use of PBO a motivational 
tool for teachers (see the beginning of this chapter), there remain a number 

_ ■_ ■ -__ :;_ \ 

of critical unknowns: \ the validlfty of the procedur'es. as a melan$ for assessing 



teacher performance 

^1 J 

performance apprais, 



; the best approach for linking the fBO process with other 
al procedures; /and clear, quantitative, systematic evidence 



__]_ ' . _ ...... _ . . _ ' 

df the effectiveness of'^such programs when designed with the'^goal of mdtivat- 

i _ ^ - . ' , ■ • ^ • * , r 

g rattier than merely appraising teachers. Systematic trials and evaluations 
pear badly needed to assess whether ndn-*mdhetary .perf dmance-by-bbjectives 
ograms, prdgrams that carefully attend to the problems noted above ^ can * . 

- - _ : - _ - _^ ^ — . " \ 

effectively stimulate improved teacher jperf bniian9e« 
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^ CHAPTER 8 ' * . 

* - c . * • 

issues After the Plan Has Been Put .Into Operation ^ * • 

' : ■ ^ * ■ . ^ • ... ■ 

Here we discuss issues rel^ating %o adequate annual funding by the school 
diBtrict, what shpuid be done about evaluation and Tevtslbn of the plan after 
It is In operation, and whether there are any special resource needs for 
operating the'se plansi > ^ * . . 

Will Adequate Funds Be Made Available Each Year? ^ 

School diatricts that introduoe a monetary incentive plan will need to 

adequately fund theta eacl^ year they will be' uhdemined* Itievitably there^ 

• ~f * _ _ 

will be temptations during periods of t^ht revenue to cut back on the dollar 

aUbcations,^ thereby festricting the'^u^ber ahd/br size of the awards. .•*Malce 
sure you have enough ^money to make merit^,.^^^ at tractive tb teachers" (Crameri 
1983, p. 35). ^ • ; , . 

Robinson (1983) reported that the Educational Research Service's 1983 
survey found lack of funds and inadequate financial incentives tb be a f re-^ 

qiient cause of the failure of teacher merit pay plans w Penn Manor (Pennsyl-^ 

... f 
vania), for example, •enCduntered problams when faced with a tight budget, 

especially in trying to narrow the field of candidates from, those who had been 



rated 6utstahdihg«f. The choice was tb divide the money into smaller amounts 

^ : ■ C5 ^ 

fbr each teacher or to harrow the\tfield further and give sizeable amounts to 

bniy a few. .In this case^ the school board increased th^ allocation, p^rmit- 

^ing ail thbse evaluated as butstahdihg tb receive the full $1,000 bonus* 

(Cramer, 1983 and Penn Manor case repbrt).' ^ ^ :: 

Many, if ^ot most^ school districts place a tbtal dbllar limit each year 

on the money available for iWards?^ This is a natural tehdehcy because of the 

need to provide a balanced budget. Others^ however ^ did hbt~8uch as Hbustbh 
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and Midlahdy Texas (see Hdustbd case report and JiRS 1983^. Where dollar 
limits are not established at the be^nning of the school year, ^he school 



district takes some risk that there will be an unexpectedly large number of 
awards. Houston felt that it, could estimate the toal cost of the of awards 



reasonably well, and in such a large school district, any dver-expenditure 
would probably not be large enough to cause a major budget problem* 

School districts that provide yei^ limited budgets for incentive awards 

_____ _ _ _ _ - _ ■ _ 

will force the administration towards the use of quotas or very small rewards, 

or bbth» This will lead to concerns (as discussed in Chapter 4) by teachers 

and the admlhistratibh that all those who perform meritbridusly during the 

year cahhbt be given adequate reswards— -to the long-run detriment of the 

plan* It may well be better for school districts that are not willing or able 

to commit adequate annual funding for the plan not to initiate a merit pay 

plan at alii 

What -F r^ 
Plan? 



It is not likely, and perhaps- is inconceivable, that a plan can be per- 
fect the first time around* Explicit provision^ for periodic evaluation/review 
seems highly desirable, even if only to assess whether the plan is accdfiiplish- 
iiig i'ts objectives and is "cbst-ef f ective." 

Some bf the plans that we examined were mbdified substantially after the 
initial version* Fbr example^ the Houstoh and Bryah^ Texas plans both went 

' i t s • . * 

!-^___"_ .» 

through a number of revisibhs^ including changes in the size of award qubtas^ 
bbhus amounts^ aiid evaltiatibn crit^la Csee those case reports^. Districts 

such as Catalina Foothills (Arizona^ , Hbustbn^ and Ladue (Hissburi) have 

' ' • i 

^ . • u-^ 

annual evaluations of their plans* 
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ERS (1979» p»85 reported the heed for cdhtihiial evaluation so that 
^obiem areas can be identified and corrected. The Cdhgressidnal Task Fdrce oh 
Merit Pay (1983) suggested that plans should be subject "td periodic review 
for refinement, improvement > or abandoment." Ladue (MissduriJ has a standing 

committee to conduct a continuous evaluation of the program and to recommend 

\ 

desirable changes. The committee includes teacher representatives. Ladue's 
committee has surveyed both current and former teachers and administrators to 
help evaluate the program (^e Ladue case report). 

By and large ^ the evaliaatidhs that we found focused on obtaining staff 
viewpoints^ ' including thdse of bdth administrative personnel and teachers. 
Sometimes these viewpoints were dbtaihed systematically^ for instance thrdiigh 
surveys using formal questionnaires^ perhaps undertaken by an outside 
consultant. €to rare occasions the districts examined changes in selected 
criteria related to their objectives* for instance teacher attendance and test 
scores (see, for exMpie^ the Seilihg^ Oklahoma^ and Hdustdh case reports). 

Overall, however, we found few attempts by school districts td compare 
performance on ^evaluative criteria after the plan began to that for periods 
before the plan begani School district administrators with whom we spoke 
often indicated that adequate baseline data were no/t available and had not 
been examined* Also, few attempts were made to explicitly look for other 
plausible explahatidhs for any changes found, such as changing pupil composi- 
tion or ecdhdmil^ cdhditidos (that could explain changes in test scores or 
teacher retehtibh rates). 

The heed for review applies td perf drmahce-by-dbjectives plans as Well as 
monetary incentive plans. Newpdrt^esa evaluatdrs, for example, urged school 
districts initiating a teacher accountability ssystem td assess the degree td 
which the system has been implemented according to the specified procedures 
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and to do this early in the school year so problems can be identified and 
corrected before it is too late (Niedermeyer and Mein, 1972)* PBO plans 
require considerable added paperwork; the value of this added burden shooid be 
periodicaliy assessed to ensure that it is still worthwhile* 

A school district developing an incentive plan should attempt to obtain 
historical data relating to the d'bjectives of the plan (see Chapter 2) that 
these data can be compared to similar information obtained after the' plan has 
been in dperatidh. Such data could include student test scores ^ teacher and 
student absenteeism figures^ as well ^s systematically collected data bh 
teacher morale and attitudes and dh parent and student attitudes. 

Such evaluation procedures have the drawback that they require special 
effort ahd^ perhaps^ special resources. However^ if the school district 
wants to be able to assess how successful its incentive plan has been and 
whether it is worth its cost^ these tasks seem necessary. 

Are There Special Actlytttes and Ra&oariies— Needed for Operating the Plan? 

Depending on the plant's design, there may be special activities and 
resources that a district will need to provide throaghout its operation. 

In the previous section^ we discussed the need for an annual evaluation 
atid review process. In addition, special teacher evaluation procedures and 
data processing may be needed. If the schbdl district mdveB to a more* struc-* 
tured supervisory^ or peer, rating prdcess than it has had (involving, for 
example^ systematic classroom observations), special training will be needed 
for those persdhs whd conduct the observations /evaluations. «(A teacher eval- 

r _ - - _ _ _ - _ _ _ ^_ 

uation system using peer ratings will have this prdbltin to an even greater 
extent since there are likely td be *mahy persdhs ddihg the ratings; all the 
peers need to be trained to achieve reliable ^ cdmparable ratings.) Training 



: ■ / 

objectives cbmponent ^ whethiBr the plan is jnbhetary (see Chapter 5) or hoix^ 
monetary (see Chapter 7). As we have noted in thpse chaptexs^ extensive 
training for participating teachers (and supervisors) in identifying 
objectives and setting targets will be needed; 

Such special training is needed not only in the initial year of the plan 
but is also needed annually for new staff. Refresher training is also likely 
to be heeded periodica4.1y* ; 

Such special training can help defuse perceptidhs of the lack of bbjec- 
tivity of. the evaluation procedures (Cramer^ 1983| p.35)« For perf dnnance--by- 
objectives plans, such as that in Hyde Park, considerable training is heeded 
to develop skills associated with setting objectives and subsequent appraisals 
(Gray and Burns, 1979, pi4i5). The setting of objectives^ including the 
selection of objectives that are both loeaningftil and measureable^ is hot ah 
instinctive activity , and is npt easy." Special training and technical 
assistance will likely be needed, tack of such help appears to have caused 
major problems for PBO plans involving educational as well as nou-educationai 
personnel. 

i s 

Training will be much less of a problem in a plan based on student test 

- n 

J, __ _ _ __ __ ________ ' 

scores and other "objective" measurements. However, as has already been 

discussed ^ pl^hs based on test scores will likely^need to provide for 

additional testir^ and arobably additibhal tests, especially tests that better 

match the district's curriculum (see the Houston and Selling case reports). 

' In addition, the district will need to provide resbwces for scoring the test 

results, and perhaps for special data processing and analysis efforts, tising 

either in-house or outside help^ It. may be also necessary to provide special 

security precautions for the testing process. i ; 
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I Finally^ It is desirable to provide orientation on the jplan to prospec- 
tive and new teachers ^ iadue teachers, when surveyed in a 1983 evaluation of 
its monetary incentive •plan, reported that it was vitally important to .orient 
new teachers to the plan (see Ladue case report and ils October 1983 plan 
description). In the case of Ladue, members of the evaluatldh committee plan 
to meet with new staff members by the fifth week >cif the school year to discuss 
the evaluation program (the Committee aJtsb requested t;hat^adm±nistrative 
personnel who interview candidate teachers describe the plan to them) 4 

' , - X . 1 

Unfortunately^ we have f ound no readily available data :as to the costs of 
these activities (e.g. special training, testing, data processing^ etc) • the 

___ ._ ._ . 

cost will depend on the school diistrict's size and its own- e^xistlng resdurqes. 
These Activities need to be planned f^r and'^budgeted by school districts. 
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How Should the Incentive Plan^ Be Integrated Wits 
Career Ladder/Master Teacher Plans 

Career ladder/xnaster teacher plans are generating considerable interest 
throughout the United States. Numerous variations have appeared ,) several 

_ , ■ ■ __ . _ V 

under the banner of "merit pay." This has led to some confusion over the 
relationship between merit pay and ^ster te^cher/career development programs. 
There has been a tendency to lump together teacher TOrit pay plans and 

career ladder /master teaser plans. In this report , we draw a fairly sharp 

" __ _ „ ■ 

distinction between the two and focus on merit pay plans, not career ladder 

plans. Career ladder plans usually provide salary, differentials based on 
special assignments liivdlvltig different or additional duties. ^ such, they 
give added cbmpehsatlbn. to teachers that have different respdnsibillties from 
other teachers^ rather than for a high level of performance for the saioe basic 
set of duties. For examplly a master teacher plan may call for teaching other 
teachers, the evaluation of other teachers, or the develbpmeht of curriculum. 

two general types of master t.eacher/ciareer development programs are 
especially important. One class combines features of both merit pay and 
career ladder plans, providing higher teacher pay grades (and increased 
stature) for a combination of criteria such as 'perf6rmah<ie ratings^ partici- 
pation in special district projects, educational credits^ and longevity. 
However, the master teacher's primary role is still teaching students, thus^ 
this type of plan is a merit pay as well as a career development program and 
is of direct concern to the subject this reports 

^ ^ - ■ ^/i - ■ 

The second major approach to master teacher, programs emphasizes the crea* 
tioh of hew job posit ions for teachers, jobs with additional responsibilities, 
extra pay^ and - perhaps!- added prestige. More importantly, such new jobs 
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often move good, teachers away from the ^nscruccldn d£ scudents and into areas 

such as curriculum development^ teacher cbunselliig aiid tralnliigy ^Cc* In gen-> 

■__ ' .__ - _\._ ' 

eral^ this type of plan does hot represent a form of whet is usuariy termed 

■ . ' • ■ , • \ . 

^ _ _ _ _ __' _ __ _ _ * " 

merit pay^ since (1) it is not usually designed to provide a direct and rela- 

t 

tiveiy immediate link between performance and pay, (2) such plans do \iidt pro- 
vide a direct stimuiuB for improving a teacher's performance with resi^ect to 



time^ 



the teanher's present job (e^gi teaching students) » and (3) at any giSrei 
such programs probably provide advancement opportunities for only a 8maii\pro- 
pdrtidh df teachers. In fact, candidates for such programs are-o^ten selected 
primarily on the basis df tenure,,' specialized skills, leadership, expertise ais 
a trains- » and similar attributes other 'than - dr in addition to - job perfor- 
mance (although, of course^ performance levels must be satisfactory). 

The primary objective of the latter type of master teacher prdgram is 
usually the recruitment and/br retehtlbh of good teachers by providing addl- 
tibhal bppbrtuhities fbr earning extra pay aqd prestige and for assuming new 
and challenging responsibilities. Improveiiiikit of student achievement is 

usuaiiv only a secondary objective^ bhe that is addressed billy indirectly j 

^ _ __ _ - _ _■ \ 

(ecg. as the result of retaining and recruiting better teachers^ having mastir 

teachers develop new curricula, ot providing special training and assistai^ce 

to other teachers). 

__ _ '-_ _ " 

A potential danger with such master teacher plans is that they may 

_ . • . _ 1 _ L. ___ y 1 

encdurage teachers who are very good teachers of students to dilute or change 

_ _ _ _ 1 

their work in order td receive higher pay — to a job in which they are not as 

good br as happy. In any case, master teacher programs are iikeiy to involve 

only a small percentage of teachers at any time and thus will be something of 

an "elite" program. (Hbw many teachers could be pulled dut from teaching 

f ' .... 

pupils to teach other teachers?) 



114 



We thas focus here on plans such as that of EVanston'^ liiinots that has 
several promotional tracks, cojaib±n±ng some features of a d±f f erentia-ted 
assignment plan (e.g. active participation in district-vide committees and 
projects), but which leaves the participants priinarily teaching ^udents and 
which makes them meet a substantial teaching perfbrmahce requirement for id^ 

cehtive pay to be earned. For example ^ Evahstbh has five teacher levels^ but 

_ _ _ ■ . _ _ _ ' ' 

teachers are still priinarily respbnc^ible- for teaching students, and they are 

required annually to achieve a particular evaluation rating to stay at that 

level (s^e case report).' King William County (Virginia) has a somewhat 

similar plan. In fact^ it uses the term "master teacher** and ^'career teacher" 



for Its top two levels (of* four)* Agaln^ however^ teachers continue to teach 
students as their major assignment, and their annual performance evaluations 



And 9 as noted t most do 



must meec specified standards. :Both of these plans also Involve longevity and 
educational requirements for promotions to a higher level, thus, though these 
plans may be considered by some as variations of master teacher programs, they 
are considered here as being primarily teacher merit pay plans. 

Master teacher programs are in general quite new, and .therefore we have . 
hot in this effort attempted to assess specific plans. 

hot appear to, fit well Ihtb the category of teacherj incentive plans as defined 
here. 

At present^ there appears to be even less information bh the effective^ 

hess of master teacher plans thah on teacher merit pay and perf oxnnahce-^by^ 

objectives efforts^ Most trials are just begtnnihg. It will take several 

years to begin to determine the impacts of such plana on teacher recruitment 

a:nd retention, and probably longer to assess the effects on student achieve*-: 

meht and abademlc excellence (because of the very indirect linkage between 

such master teacher plah^ and educatidtlal outcomes). 
1 
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^ CHAPTER 10 

What Shbiid Be the Pole of the State fioverninent? 

f - . ' ' 

State polices ciS play a tnajbr role in encouraging or discouraging school 
^ -district incentive plans. ' St^te laws requiring or prohibiting collective bar- 
gaining, have substantial implications for school districts Wanting to intro- 
duce, an incentive plan. V^cBool districts covered by collective bargaining are 
likely to find implemehtation n^re difficult, with more obstacles. States inay 
also have specific regulatioSs regarding teacher evaluation practices. 
• Teacher evaluation practices are also affected by the presence of collective 
bargaining, which can have substantial effects on the ability of scfeool 
districts to revise their teafcher evaluation process. (The early partid^ . 
pation Of teacher representatives in the development of Incentive plans and 
teacher evaluation procedures can. however, lead to a sounder and more durable ^ 
progsram.) State laws regarding tenure and compensation of teachers can also ' 
hav^ substantial effects on the form and timing of a plan. For example, as 
noted in Chapter 3. state laws concerning the treatment of salary .reductions 
as demotions can inhibit districts from providing for performance^based sala^ 
reductions in their plans. Each school district will , of course, need to 
check its own state laws and regulations and determine what constraints exist 
aSd what might be heeded to alienate such constraints. 

Th^re are other less legalistic roles that the state gbve^nments can 
potentially play. States such as Virginia and Florida are undertaking efforts 
to either develop teacher evaluation Ins tnioehts or to provide seed money, 
such as -in Virginia, to pilot test incentive plans. A state government has 
the Advantage offing able to bring to bear iore Resources in a more concen- 
trated way to identify sOuhd incentive plan practices. Unfortunatelyi ie ari 
hot optimistic th^t this will occur. We dd not see much Sign of ^ste^tic 
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>'4^6*tg to encourage\th5rbagh and comprehensive ieei^S^d^e^lu^tlon of 
•fllans within- individual states so that, other school d±stri6^ can learn fully 
•^^om such experiences. W state process seems more likely to ^ther latch 
^■^^t^.a pSi^icular incentp^proach or to let everyone go thet^ own way 

Without adequate provision f brN^o ordinated statewide evaluatiot^ so that 5t6ef 
-,v|thool districts can obtain usefW information from thoSe ix^^iences. 
' 1%.^ Another potentially impbrta^ role for states is in the area of studeni 
'i^es^lng. if state agencies couldYevelop appropriate annual achievement tes 
"^ibfiriJtE, district use. this couli greatly ease district problems in; assess- 



ts 



ihg &t 



s:tAlfcit learning progress fbr incentive purposes, 



, . ^F,.lnaliy, States could provide technical assisrance and training for 

School- di^rictiiersonnil. is discussed In previous chapters, there krfe 
• ; ouieroUs 'technical and procedural issues bn which individual schbbl district. 
-^o^^i^iit urn technical assistance or training-end pSobably fr«» both. 
' would require that a state have staff who can give such assistance and 

tiaittlng'or that it make available other resources such as university 
persotmet* - ' 
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Bryan Indepehdeht School District; 

Bryan, Texas ' ' 

) * 

The School Disttfct ; 

The Bryan school district consists of 12 schools (9 elementary, 2 Junior 

M _ ; 

high, and 1 senior high). ^ The number of pupils lO^ABO^ and the number of 

teachers, 535. The teachers are tiot uiiibtil^ed but belong to several prbfei*^ 

sibiiai assbciatibxis such as the Assbciatibh of Texas Prof essldnal Educators* 

Teacher salaries stax?t^^_ $13| 110 going to a maximum of, $22,660 for a B.A. 

_ _ . , — _ _* . . _ . _ 

F'or kh M.A. the starting salary is '$14,080 going to a liiaximuBi of $23,220. 

Description of Plan ^ 

, . 

Bryan's merit plan began in the 1977-78 school year. ^ The plan currently 



consists of a bonus given for "superior'* "^eacbers^ a bonus *^ over-and-arbove the 
regular salary, this plan covers all teachers of ail subjects in all grades. 
Only teachers are included* . * 

In addition to this merit bonus^ the district awards "merit retention 
increments" (MRI)» a program that started in 1981. If any teacher ^ete four 

i 

merit bbnuses (nbt necessarily in cbnsecutive years^, the teacher receives an^ 
addi'tibhal award--*a special salary itlcrease^ih the fifth cbntract peribd. 

Each year the school board decides what percentage of the teachers can 
^t a merit bonus and the size* All teachers receive the same size bonus and 
MRI amount. These amounts have been ais follows: 



Year Quota Merit Bonus Amount MRI Amount 

/ 

1977- 78 20 percent $600 

1978- 79 20 percent 600 

1979- 80 25 percent 600 

1980- 81 3b percent 600 

1981- 82 3b percent 6bb $600 

1982- 83 30 percent 800 800 ^ 

1983- 84 35 percent ' 1,000 ' 1,200 



B - 2 

4 

I 

The quota percentages are applied to each of the 12 schools ^ but some flexl- \ 
" bility is permitted. If one principal feels that there are more than 3^ / 
percent "superior" teachers in- his school and another principal awards bonuses 



to less than 35 percent, then the first principal can hand out bonuses abovg 

■ ' - ' ■ • ^ \ ^ 

the quota figure. ^ 

— . - — _ _^ — _ _ • _ -\ 

In 1982-83* 152 of 533 teachers (29 percent) received bonuses. Af^rdxl- - 

mately 40 percent of the ierit pay fecipiehts in 82^83 got HRIs. (That Us * 40 
percent of tRSir teachers had won the bonus four times.) V ^ 

The merit pay or bonus is provided quietly without any publicity. In 
fact, the bonus is included with the regular pay in the same paycheck. A . 
teacher can go to the principal and see the 'list of bonus winners. (The dis- 
trict holds ^.a' ceremony in the civic auditorium for other awards: longevity 
awards as well as award^ made for the "outstanding^ elementary school teacher 
oi the year" and the "dutstknding secondary school teacher of the year i" Sach 

of these two teachers receives a check for $700^) , 

» ___ __ \ 

Teacher JEvaloation and Selection of Winners . i 

the heart :oi Bryan's teacher assessment is classfbbm observation^ ClasS'* 



room observations are done by: the principal* the assistant principal, the 

_ _/ , -_ ■_ •■ _ -_ ■- . 

curriculum leader from the central, office^ and the de|)artmeht-/head or 

chairman. Each oE^erves the teachers separately during visits scattered over 

the yeari the first visit is by invitation off the teacher. A^|f^^ that, it is 

unannounced i** Where differences o^cur between the difJerent aSsessbrs^ it is 

the responsibility of the principal to work it outi As each observation is 

completed, the findings are discussed with the teacher* \ - 

The ratings, from 1 to 5 on each c^^terion, are combined by a weighting 

system into ah overall s|cbre. The criteria include: personality factors • 

(e.gi enthusiasm^ sense of fair play* etc*}; social and prdfessional factors 
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(e.g. attitude towards teachlag,, loyalty to school; etc.); classroom organlza- 
tion and root Ine (e.g. linpartlal treatment; of students); teaching techniques 
(eTg. skiii in'.questioning); v|hether the teacher ^is active in improving the 
V profession; and teacher involvement In coiSuSity activities. The ratings for; 
each of the criteria are not anchored i The ratings, therefore, are highly 
subjective. 

Principals may also use other information to help them make their rat- 
ings, such as peer observation, pupil or parent assessmehti assessient against 
pre-set targets, and records. The district does not use student achievement 
(e.g. test scores) as a ciiteridn because it feels that some teacheri may have 
slow lea-rners and others might hot. Pupil or parent assessments are used on a 
very informal basis,- e.g. a pupil or parent can come into the principal's 
office and give an assessment of a particular teacher* Records on absenteeism 
are incladed in tlie evaluation. 

The principal keeps a folder oh .^ach teacher* The folder contains 
records about the teacher (e.g. absenteeism), the findings, on the observa-^ 
tlbius, the goals set by the teacher, the teacher's self-evalua tibh done in 



September (based on the last year's work), and a fonaal evaluation conducted 
"by the' principal in February^ At the end of. the year the principal examines 
the folder and nominates recipients • 

The^ principal has to justify his uotninatidns before a central screening 
committee, coSsisting of the superintendent , director of personnel, and th^ 

__ _•>_ _ ' 

d4.rector of instruction. This procedure was established to miSiiize the like- 

" _ _ _ _ _ __ ^ . 

iihood that pelS^onality conflicts would affect the awarding of the bonus. The 
screening committee Sot only exami^es^iJie-^r^^^ why the 35% have been 
nominated, but also, why the other 65 percent have not been nominated. The 
teachers can appeal to the. principal or even to the central committee, but no 
3ne has yet appealed to the central committee i 

ERIC 



It took the district one year to plan and prepare this program. The 
district used Input from businessmen » PTA m«(mbers» administrators^ and the 
teachers • The director of persbhhel obtained the assistance of the teachers 
in first overhauling the old evaluation system. After that, the merit bonus 
system o^s added and the teachers were asked once again to review the evalu-* 
ation criteria, there \ntB no formal participation from the teacher's associ-* 
ations although their representatives were on the committees that developed 
the plan. There are cdntinulng efforts to explain the plan to hew teachers. 

the observers themselves arc trained regularly by prbfeksbrs from the 
local university. This tralhlhg is held three times during the year. 

The board supports this program but leaves the details to the superln-* 

- - - - c 

tehdeht. The' boards however » determines the quota and dollar amquhts. The 
principals hbmihate the candidates. The remaining work is done centrally* 

Evaluation of the Pl^ 

there has been no outside^ formal evaluation, but they continually review * 

_- _ _ _____ __ _ _ __ _ _ _ ^ _ _ ^ 

the plan themselves ("semi'-fonnally"). For example » they have a workshop" 

planned for next July to exatiilne the merit pay plan. 

According to our respondent it has affected the teachers in two ways: 

(a) they appreciate the recognition of merit » and (b) it helps borderline 

._. _ ■ .... . .. . ' \ . . ^ • 

teachers Improve, the teachers that rec^'ive the bonus are "tickled to 

death." the others would like to drb|> it» bUtr there havej>biln nb fbrmal 

complaints. Apparently » the teachers generally like tHe plan. It has helped , 

_ _ r-> - . - " ' - ' - — , . 

accbrdihg tb the respondent » tb recruit teachers because teacher applicants. 

are generaJJLy excited abbut merit pay. 

In 1982-83 the cbst pf the merit play plan was about $169»750» abbut $320 

per teacher fh the district'. This is the actual cost of the*bbhu8 (accbrdihg 

to the respbhdeht there Is nb appreciable coat for administratibh). 

. 137 - . . Vr- ^ 
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Since there is no special evaiuacion work undertaken for the merit bonus 
(teacher evaluation would be done anyway), the only additional cost (over the 
extra bonuses and salary increases) is that incurred by each principal to 
justify the homihatibhs before the central cdmniittee aj|d the additional 
computer effort required to add the merit bonus to the appropriate paycheck. ' 

The state government has had hothihg to do with the plan* The district 
found no state laws that have either raised obstacles or encouraged ^the plan. 

Suggestions to Other s , - 

The respondent provided two suggestions: ' ' ^ 

!• The school district should have a good eipioyee benefits package 

(salary » insurance, etc.) already in place for every teacher. The 

merit program should not be part of the regular benefit package but 

should be the "icing on the cake." ; . 

2. ■ School administratdrs cannot sit down in a central office and impose 

• " ' 

such a plan on the staff. They must get input from the teachers for 

such plans to succeed* A school system must have the teachers help 

plan the program. 



Iw ERS, "Merit Pay Plans for Teachers: StatH^s and Descriptions," 

Educational Research Service, Arlington^ VAKi983 pp. 49-50; lOS-lO?; 
122-127. ^ 

2. Telephone interview, Decraber 1983 with C.B. McGowh, Director of 
Personnel, Bryan independent School District^ Bryan, Texas. 
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Catalina Foothills School District 
Tucson » Arizona 

The School District 

The district consists of two elementary and one junior high school, 
together covering (So high school). ^ The Fall 1982 enrollment was 1,425. 
The starting salaries for new teachers are the highest in Arizona?. ^^^^ 
steps, however* are about avetage for Arizona schools. 

DescriptiQP of Plan 

Cataliha's plan provides annual awards based 'on perf oriSiahce ratings inade 
by the principals and assistant superintendeSti The ratings are based pri- 
marily on classroom performance, i.e. process characteristics * using a teacher 
evaloattpn instrument with five ma^br categories, in 1982-83* 35 of the 85 
eligible^ teachers employed by the District (41 percent) received merit awards. 
The lowest award value was $300* the highest was $800, and the average value 
was $555. The plan began in the 1980-81 school year. It is now in its fourth 

year .(83-84). ' - 

A unique feature of the plan is that the teacher, with the principal* 
seiectl the particular form of the award* cash being only one form, the 
awards chosen by teachers include such p5ssibilities as attendance at profes- 
sional conferences held outside the state and the purchase of instructional or 
enrichmeht materials— such as a micro-c5iputer, a telescope, an aquarium, or a 
cameSa^kiiii ti> ^ uked by%"e teachers inj ciassroois. Thus far, about 30% of 
the awards hav^ been in cash, 50% have been used for participatibn in 
conferences or workshops* and abodt liZ for classroom pro jects/materiali. 

the prtnctpil, or sometimes the assistant supe^iSteSdent* makes classroom 

■ n _ _ _- 

observations and ^atiigs using a standard seating form; For teachers with 



139 



e - 2 



tenare this Is done at least once a year wlt^ an interview shortly thereafter. 
On each of the five primary rating categories (assessing student needs » plan'* 
liiiig and implementing instruction i student-^-teacher relationship » evaluating 
the instructional program, ^ahd prof esslbnalismJii^a teacher^i^ rated either: . 
exceeds standards, meets standards, needs to improve^ pr unacceptable* No 
quantitatirve score is developed from these ratings, nor is an overall rating 
eKpiicitly assigned. Teachers with any one of the five categories rated in 
the third or fourth level (indicating improvement needed) generally do not 
rei^eive awards. Completely explicit criteria have not be^n made c^icial ^ 
policy, though in the past two years award winners have been those who have 
received ratings of "exceeds standard" in at least two of the five categories 

with no ratings of "needs improvement." 

— — _ 1^ 

The superintendent annually requests a specific amount of funding from 

the school board for the merit awards. Thus far^ the school board has pro^ 

vided the amount requested. The teachers to receive awards are identified in 

May at the end of the schobbl year. The superintendent makes the final decl^ 

iions based on recommendatibas from the principals. There is ho appeal 

prbcess. The superintendent's decisions are final* The principals then 

/ ' : % 

notify the teachers and dtscuss with the teacher the particular form of award 
the teacher would like* The awards are given af'ter the start, of the next 
fiscal year (July t) with all expenditures being spent by approximately 
February of the next calendar year. A teaqher, for example, could obtain 
cash, or attend a conference during the summer, or delay using the award until 
later in the school year. 

The teacher evaluation instrument was introduced at the same time as the 
incentive plan but had been under development before and would have been in-* 
trodiiced regardless of the coincidental implementation of the merit plati. The 
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instrunseiit was developed Jointly by teachers , school district administrators^ 
and the University of Arizona* It was developed after examining the research 
on teacher classroom characCeris tics that appeared to be related to st^^nt 
achievement* The teacher's assbciation, an affiliate of the NBA, did not 
participate in the fbrmulation of the merit plan* After implement at ion » 
recommendatibhs given by individual teachers have been cbhsidered and used tb 
modify the plan* 

Ml teachers are covered by the plan with the same plan and same pro* 
cedures (e^gi the same teacher ;evalaation instrument) being used for all 
teachers in both the element^^ and junior high schools* Administrators are 
not included in the plan* The plan focuses on individual , not group, incen- 
tives* 

The district's policy is not to release the names of specific individuals 
receiving the awards, though this information does tend to become known among 
the teachers* . v^. 

Evaluation of the Plan 



ERIC 



No evaluation had been conducted of the effects of the plan on student 
achlevemeut such as oh standardized test scores* The test scbres for Catalina 



Foothills pupils, however^ have risen considerably in recent yeata«^ut the 
superintendent Indicated that several other major changes th^t bccurted during 
this period could also account for these increases* No attempt has bei^h made 
to obtain evaluations from the parent^ or pupils regarding the plan* Hbwevef, 
after the first year, the district surveyed the 27 teachers who received / 
awards and subsequently interviewed six randomly selected participants and six 

\ _ _ _ , __ _; ^ ,_ 

randomly seiectjed non-participants-— to obtain information on their attitudes 
towards the program* Only minor dissention over the plan was reported in 
those surveys* (Substantial concern did arise in one school in which thie v 



• ; / ' 

principal had distributed a rantced list of teachers' oame^* This procedure 
has beenN^stbpped*) After the second year, the teacher's union surveyed the 
teachers. About 50 percent of the approximately two-thirds of the district's 
teachers who responded express^v^approval of the prbgr^. , the major concerns 
of those who indicated disapproval were that they did not know the selection 

_i ; , , _ . _ J ._. ' 

criteria, they did not onderstand the purpose of the program, and they felt 
that the program money should Instead be given to all teachers qr used to buy 
materials for all teachers. The administration at the end of the second year 
surveyed first-*tlme participants* The administration's interpretation of 
these Is that the teachers were motivated by the plan* . 

The administration has pointed to examples of positive actions undertaken 
by teachers after receiving their awards* For example , a. third^ear English 
teacher In a junior high school oh returhlhg from ah award cbhferehce I'tUy-^ 
Boston volunteered to develop a program to evaluate writing in the ele^enta^ry^^ 
schools* 

The cost of the program has been primarily the ^oont distributed for 
awards-— about $16^606 In the first year^ $15^666 In the second, and $20^600 in 

the third. An average of ab^t $236 per teacher in the district was paid in 

■_ __ i_ ■ J_ 

FY82-83* Because of the, simplicity of the teacher evaluation system^ no 

special data processing o** analysis is required* Because the instrument had 

^_ _■_ ' • , 

already been in development at the time of the plan , no additional cost was 

incurred for generating the teacher evaluation Instrument* However, another 
school district might need to invest funds in developing an instrument satis- 
factory to the admlnlstratlbn and teachers before initiating the plan. No 
special testing of students is involved, so no special testing costs have been 
required* The observWiiSn time of teachers had already been at least partly 
required by a state mandate and little extra time has been necessary since the 
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formal procedures do not require extensive amounts of observation time. The 



scbbbls average tender 30 teachers per school and principal observation time 



has hot 9 thiis far§ been perceived as being a special burden* 
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Evans tbti Public Schools ^ 
Evaxistbn^ Ullxibls 

The School District ' 
^ The Evahstoh School District (Coble County School District 65) consists of 
9 elementary » 4 junior high schools^ and 1 school for multiply students (no 
high schools). are approximately 6,123 students with 438 teachers* 

the District has had merit pay for teachers since 1959. the plan con-- 
sists of five tracks. On any one traclt, teachers move a step for each year of 
service* they move to higher tracks based on performance, education, and 
"outside" activities. Flrticipation in the plan is voluntary* 

Track I is for teachers at the beginhihg level* To move to Track II 

- _-- ~_ " "_ " ^■ 

teachers must either complete a master's degree or oh their annual teacher 

performance evaluations have either three ratings of "exceeds expectation" 

within a five-year period^ or six years of either "meets" or "exceeds" 

expectation* (These are the two highest ratings; a teacher may also be rated 

"heeds to improve" or "unacceptable*") 

To move from Track II to Track III a teacher iiast complete i5 graduate 

semester hours, be an active participant in building ^ department or 

_ _ ■ _ _ _^ _ _ __ ■ 

distribution committees ahd/or professional organizations for at least three 



years while on Track 11^ and as before, have either three ratings of "exceeds 
expectations" within a five-year period or six years of either "meets" or 
"exceeds" expectations* 

For movement from t tack III to track IV a teacher must have completed a 

<. . ! f. ' • ' 

master's degree, had active participation in^oth building level cbopoittees 

< a 

and district-wide educational activities for at least three years while on 
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Track III, and have had four annual ratings of "exceeds expectations" within a 
seven-year period. 

• To reach Track V a teacf^er must have completed 10 graduate semester hours 
beyond ¥ master ^s* degree, had active participation and a leadership role on 
building or di«trict-wide committees and projects, and have had four annual 
ratings whil^ on Track IV of "exceeds expectations" within a seven-year 
period • v. 

teachers on Tracks IV and V are given a special re-evaluation at least 
every three years* If a teacher receives an evaluation other than "exceeds 
expectations", the teacher and evaluato* are required to initiate steps to 
return performance to the meritorious level. If a teacher receive a "needs to 
improve" rating two consecutive years ^ additional cbmpensatidn is denied, 
freezing the teacher^s salary. 

Teachers are annually evaluated on their perfbrmahce in five job target 
areas: 1) teaching skills; 2) classroom environment; 3) cbmmuriicatidn; 4) 
interperi^bnal relations hips; and 5) professional cbhtributibns. The evalua- 
tions are dbhe by the principal or teacher^s supervisor. There are three 
different variatibns bf the evaluation procedure based on tenure status, the 
previous yearns performance rating, and whether the teacher is due for ah "in-^ 
depth" evaluation. 

The "in-depth" evaluation is given to non-tenured teachers yearly and tb 
tenured teachers fevery three years br as determined by the principal. Approx- 
imately 1/3 of tenured teachers i^eceive in-depth evaluations each year. At 
the b^.ginning of the schobl year^ the teacher and evaluato^ set job targets 
jointly, and agree on what help the teacher can expect from the evaioator in 

meeting the job targets, and on the way in which performance on the objectives 

/ ■ ■ - ^ .._ . . _ . 

will be measoredi One of these targets has tb be bn student performance* The 
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« district tries to keep the numbei: of bb;jectlves to 2-3 per teacher » believing 
that more may dilute the teachers^ effort acid overwhelm them. The evaiuator 
makes a minimum bftthree bbservatlbh visits per schobl year. At the end of 
the year the evaiuator and the teacher prepare written narratives' utilizing 
the bbsexrvatibn material and any otlxer experiences relating to- the five per- 
fbrmahce areas. A final conference is held to assess Job target achievement* 
At this tim^ there is an exchange and discussion of the written narratives tb 
appraise performance; Modification of narratives can take place and then a 
final narrative is prepared by the evaiuator. The evaiuator cbmpletes an 

i 

."Overall Evaluation of Performance'* fbrni* 

In the "standard evaluation prbcedure"i teacher ratings are made bused bn 
the achievement of job targets^ bbservatibhs by the evaluatbr ^minimum bf* 
one}} and; the final narratives. The "standard modified prbcedure" is used for 
teachers whb have "exceeded expectatibns" bn the previous peffbxtsance ratings 
The prbcedure is the same except it omits the written narrative required in 
the performance appraisal section^ Use of the latter procedure can be ex- i 
tended a second year if tfte teacher's performance remains at an exceedingly 

high levels these two procedures are used only for tenured teachers. 

_ _ _ _ _ _ _ _ _ » 

Teachers have the option to apply for moveiaent to a higher level track 

through evaluation by an Advisory Committee rathex tha^by their principal. 

In this case a team of five, consisting of three teachers and no less than one 

principal, will evaluate the teacher. This group is randomly selected. The 

teacher writes year-long Job targets and aii extensive liarratlve describing and 

documenting achievements on the goal areas. Members of the Advisbiy Committ^ 

each write an evaluation of the teacher and make their own recbmmendatibn for 

track movement. ) * 



A teacher may request a conference with the superlhtehdeht after which 

I _ • ^ \ 

the teacher, may submit an appeal in writing* The Superintendent refers the 

appeal to the Advisory Committee whose decision is final. 

There are no specific guidell^nes to evaluators as to the relative impor-* 

tance of each of the ' evaluation procedures (Job target achievemexit , evaluator 

bbaervatibnsi etc*). or hbv to combine them into the one overall rating. This 

is left \o the evaluatbt. To obtain some cbnsistency ambng eValuatbrs the 

administration has recently held meetings with the principals tb discuss the 

performance levels that might be expected of teachers given the various 

ratings i , ' ■ , ' 

^ the district surveys parents on their ratings of a namber of aspects of 

^the quality of teaching and for fourteen specific subjects* This is done at 

the time of the Spring Conference with parents* tabulations are made by < 

school as well as for the district as a whole* the results^ however, are not 

_ . _ ^ _ _ ■ _ , . . _ ._ !. 

used as part of the formal evaluation of ±iadividuai teachers but may^ be used 
by the principal as infbrmatibn in determining the overall ratings for 
teachers* The district has surveyed parents for approximately five yearsi 

The Tracks have pre-set dpllar ambunts fbr both vertical (longevity) and 

/ - _ _- - _____ 

horizontal (meritbi^ibus) mbvemeht* The 1983-^84 salary range in Track I, for 

example^ is $16r2*S-23,061 while the range fbr Track IV is $21,111-32,642. 

The maximum salary currently is $36,213, the tbp bf Track er The merit 

increases are substantially higher than the Ibngevity increases: longevity 

increases for 1983-84 range from about $300 to $1*000 #hile mbyements acrbss 

tracks vary from about $1*000 tb almbst $3*000* 

_ ^ , _ 

Approximately 101 of the teacHers receive awards* each year but bf the 20- 
•J I ______ 

25% that apply. Eleven of the 430 teachers were awarded with merit pay in 
« 
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Evaluation of the Plan 

teachers have accepted the prograin/bn the whole* ^the personnel director' 

r 



said there had not been a najbr problem as yet nor a grievance regardt-ng frack 



inbvexnent, ./ 

■ - -_ _ ■ _ ,_ _ / _ ^r- _ 

Although teachers have two lorias of evaluation to choGi^e frbni^ the ndst 
frequently chosen foiri has bsen the principal as Valuator. I The committee 
evaluation method is considerably more difficulty requiring ^xtra effect 



prepare the report. ^ 

The plan is re-evaluated every two years ^ but there has been no formaj 
comprehensive evaluation of its impacts^ Changes have been made^ based^n the 
reviews^ but these changes have been minor and have not af fected^tb^^ basic 



X.'- 



framev76rk of the plan* Teachers are involved in the review proce'^« 



There is no systematic testing of student learning as part ^f^ihe pldl^'^.^ " 

■__ . __ __ ._• _ '''.v^'".-/' ' J ii' .V,^ 

(District students have achieved above the national nbrins bn stl^eht 

J --Z • ■■ \-/- r - 

achievement tests^ but the extent this has been ^ue to the p3?^/ls hot kii^wn.)^ 

-_ _ _ _ _ _ . _ _^ J^'^L^ _\ : . 

Thie persbnnel director believes that money is a motlv^Clbnal factor for t 

_ - _ _ ■ ' ^ _ 

teachers^ and the teachers In Evahstoh take prxde tn tfaetf track. rSe noted 

' : _' _ ' , 

that there was never any problem in gettiitg teachers Involved In putrof**' 

. . ' *' 

classrb6m'^ctlvlties~they are willing to serve.- . k 

. - ■ ■ -V.— ,--- . - * -^.^ 

The but-bf— pocket costs of the plan are priioarlXy those for Imcreased -' \ 

______ '_ _L :J __ - . ' _ 

performance-based salaries^ For 1982-83 this adib^teld to approximately^' 
?20,240 for about 430 teacherf » about §47 per teacHer. ^Nbte that only abbut 
20% -232 of the teachers apply fbr the plan each:^e^r.^ . 



1^' 



1. Telephone inteririws with Ronald fii ^<ikert, director of Personneli 
Evanston Pttb^Jc^ Schools i^. and Id^H^p^^'Ass^tant to the 



Saper±ntende»£7 EVanstoif Iliino^P^t^^^^^ - , 



c - o 



Varioos descriptive maceriais on the plan provided by the Evanst&n 
school district. : 



3i ERS, "Merit Pay Plans for Teachers: Status and Descriptions", 
Educational Research Service^ Arlington Va, 1983^ p« 35* 
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Houston Independent School' District 
^Houston", Texa^ 



^^he-School District * ' " 

The Houston. Independent School District has over 1S|6|000 pupils » about 
9|300 classroom teachers^ and 232 schools ( 169; elementary » 35 junior hlgh^ and 
28 high schools 5 • - 

_. 

Descriptloa _jgiatt V; • • 

Houston's ."second Siie plan" began in the 1979-80 school year. '^It Is now 
in' its fifth year (^3-84) i In the first year > ^approximately two-thirds of the 

_____ _ . _ :__.__ _^ _ 

teachers received awards. In subsequent years this fell off somewhat to bp- 

V * _ __ _ — _ _ _ ' 

tweeti ab6ut one-third, to one-half percent. The Infonnatibn on this Is not 

clear as the school district Inf drmatlbn system does not provide this data. 

(It dbes^ however > provide infonnatibn bn how many bbhtises of each type have 

been earned^ but hot hdw many di;ffereht teachers have earned one or more.) 

The plan has three highly distinctive characteristics: 

1. It Is highly structured and objective. with specific quantifiable 
elements that directly determiue who gets the awards and their ' < 
ainduht--^asically a "formula" apprpach. : " 

2. Student test scores are an important pftrt of the plaui and 

3^ the sttadent achievement bonus is a group rather than individual ^ 

» i 

' ■ i ' ' ' ' -5 

incentive with the school being the award unit (ail teasers in a 

school receive awards)^ 

there are six elements on which teachers (:an obtain bonuses* ("stipends"): 

i. TjBachlng ^t schools with* a concentration of educationally disadvan- . 



taged students; ~ v - > 

2. Teaching assignments in subjects with crlLldal staff shortages such 
as secbhdary science ^ secbhdary mathematics ^ bilingual education^ 'and 
special educatibn. 



34 Completion of additional qualifying education. | 

A. Teaching at a school labeled as "unique," e.g., having special 



students or a nev campus. j * 

- - -- - -_ - _ _ _ -\ " 

5. Exceeding a baseline teacher attendance requirement. * _ 

6« Teaching in a school assessed as having made "outstanding educational 

progress ^OEP)". 

A teacher can receive a bonus for one or more of the above elements. For the 
purpose of this report, the first four elements are not considered "incentive" 
pay and will not be discussed further herei 

To receive any of the above awards, the individual teacher must meet all 
of a number of prerequisites. Of particular concern here are three of these. 

The first is a requirement that the teacher achieve a rating of at least 
3 on a scale of 1-4 in a teacher evaluation rating. This teacher assessment 
is made by administrative persdnnel such as a principal ^ ihstructlohal super*^ 
visor, or assistant superintendent ^ usually in February or Harch of the per- 
formance year.* The rating is of a number of factors cdhtaihed oh a teacher 
assessment ihstrumeht. A.scbre of below 3 bh any factor ^^hs that the 
teacher needs improvement and thus is hbt^ eligible for any award, even if the 
teacher is teaching at a schbbl that receives an OEP awar^. flxis teacher 
assessment screens but apprbximateiy 5-iO percent of the teachers, though a 
structured questionnaire is used^ this is the one element in the process that 
im^olves subjective judgment rather than objecti^ve criteria, the teacher 
assessment procedures were a standard practice at Hdustdn before the incentive 
plan was implemented. It is during these tea^cher assessments that attempts to 
identify specific improvements in individual teacher perfdrxnahce are focused ^ 
rather than in the merit pay plan. ' - , 
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The secbnd relevant prerequisite is a minimum attendance requirement. It 
was originally set at ten or less days of absences during the current school 
year or aS average of five days or less during the most recent three years, 
& the second year of the plan, this was' reduced to five days during the 
current year. Because of this, the number of bonuses paid in the second year- 
dropped off considerably. The district is considering returning to the ten-day 
target for attendance as eligibility requirement, rather than five. The 
five days may be too restrictive. 

Beginning in the fifth year, 1983-84, an additional prerequisite has been 
included. Teachers now need to pass Houston^s new basic skills proficiency 
test'. A teacher will need to take the test once every 7 years. The District 
staff estimate that the new requirement will reduce the number of awrds by 
approximately 25 percent. 

If the teacher passes the baseline requirements, and has less than the 
minimom number of unused absence days, the bonus is increased approximately 
$100 for each day of absences less than the target (this would mean $500 for 
zero absences). , — 

The "outstanding educational progress" assessment is based on the exten- 
sive standardized testing undertaken by Houston. The lowa Test of basic 
Skills has been used for Grades 1-9; the Test of Achievement and Proficient 
(TAP) has been used for Grades 10-12. (There are two or three forms of the 
tests which are rotated from year to year. However, it is conceivable that a 
teacher could remember the questions from one year and teach the class to, 
those questions. The administrative staff believes this is unlikely.) 

The analysis of the scores to identify schools meriting an OEP school is 
complex, in the first year the average gain at a school for all students was 
calculated^ and the resulting school averages were ranked within individual 




groups of 8chbbl8--w±th the groupings based on each schocU^^^propdrtlon of 
free and reduced lunch students. Regardless of the magnitude of the gain 
score, the top 50 percent of the schools received OEP recbgnltlbn. 

There was considerable coaplalnlng by teachers of limiting awards to 50 
percent with possibly meritorldus schools hot thereby being eligible, in the 
third year, a statistical regression equation approach was introduced in which 
ah expected score for each school was deter^ned based on several character- 
istics of the school^ such as the free and reduced lunch rate, student mobil- 
ity rate, proportion -pf special students, previous year's achievement data, 
and teacher turnover rate. The actual test score for the school was then 
compared to the score estimated (predicted) by the equation. Teachers at any 
school that exceeded its predicted target could receive the OEP bonus. This 
I change not only eliminates the "quota" but also shifts from having schools 
compete against each other to competing against one's own past perf brmahce. 

Because the teachers found this procedure too difficult to understand^ 
Houston, beginning in 1983— 84» has simplified the calculatlbh process ^ drop- 
pi hg >the regressibh eqtiatibh. . The schbbl's average gain score (e.g. the gain 
frbm the secbhd grade last year to the third grade this year) is compared to 
the school's gain score for the previous year. If the average gain score haci 
Improved by at least one mbhth^ the school^ becomes an OEP schooii 

the OEP bonus was $860i It has been increased for 83-84 to $1,000 to put 
more emphasis on this component of the plan. • 

Each school that is an OEP school for the year gets a flag that it can 
fly for the whole yeari Ihxs provides an indication to the co'Timunity that the 
school has been an outstanding educational progress school « 

There is an appeals process. Thus far there have only been a small 
number of them, perhaps 20 per year. The Appeals Review Board cdhslsts of 
administrative personnel bxily. 
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The dlitrlct's teachers have representation from NEA^ AFT» and a Stkte 
assbclatibxi. Hbweveri Texas does iibt have a law permitting collective bar*- 
gaining y and the teacher association role' has been quite limited in the plan. 

Teacher reactlbnsi however^ have played Impbrtaht roles 'in changes made tb the 

*. 

plan* 

the awards for teacher attendance and outst^mding educational progress 
are paid in the fail of the school year following the school year for %^ich 
they are earned, the program covers only teachers » no administrative or 

- .r 

clerical personnel. ^ 

Participation is voluntary; teachers must formally apply for their awards. 
They can volunteer up to the time when the awards are to be paid out and after 
all data have become available. It is ^nbt known whether any teachers that would 
have been eligible, for one or more awards have n<^ bothered to apply for them. . 
(This voluntary aspect prbvides a face-saving element; teachers that did not earn 
any award could then say they did. hbt bbther to apply ^ leaving an ambiguity as tb 
whether they wbuld have received one had they applied.) 

the State's role in this effort has been small. The district found no 
obstacles from state laws when it began. The lack of a collective bargaining 

^ _ _ _ _ _ _ A 

law made it easier to implement the plan. Db state dbllars br technical 
assistance were used^ : . \ 



TVe school district undertook a formal evaluation of the plan after two 
years. It found that ':eacher absences had decreased from an average of 9.0 
days the year before the plan started ('78-'79) to 7.6 days in 1980-81. More 
recent schbbl district reports indicate that absences averaged 5.5 days in 
i982-83. Teacher turnover decreased from 23.9 percent to 20.6 percent (to 
13.9% in 1982-83) Teacher vacancies decreased frbm 613 to 376 (and tb 221 in 
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1982-83) with critical staff vacancies decreasing from 251 ' 186 (to 87 In 

1982). The data on student achievement^ however^ does not dhow an appreciabie 

imprbvemeht over the '78-' 79 year^ the year before the start of the plan* 

Teacher attitudes were found to be quite mixed | with hbn-recipiehts of the 

awards consistently negative* There have been ho formal evaluations since the 

end of the secbna year* There has been no formal attemfLt to obtain feedback 

from the parents :6r the community* The.distflct has hot received any ' 

^ 

complaints from parents* 

Administrative staff have not observed any specific changes in teacher 
practices (this is^ a very large school system)* The tuzmover rate and absen*- 
teeism have improved substantially » and vacancies in special fielas have been 
easier to fill* It may be^ however » that economic conditions over the period 
since ihtrbductioh bf the plan have been a major cause of these improvements* 

Principals^ after finding their schbbls were hbt included as an OEP 
school, have oh occasion called the plan's administrative staff tb Inquire i^y 
and to obtain suggestibns as tb what they can db abbut it* Because bf the 
aggregate nature of the test: scores^ central administrators » however; have hbt 
been able to provide concrete guidance to Individual schools* The schbbls (as 
before) have access tb the details bf the test scbres and ca^ use that infbr*^ 
mation to find out which grades^ irtticb classes i^lch teachers, and which 
components bf the test were particularly weak in their schools* it is not 
clear, however, that there is any systematic attempt to exraine the test score 
data in detail tb prbvide suggestibns fbr future improvements* 

As hbted earlier, the teachers expressed early concern about what was a 
qubta system fbr OEP awards* The latest procedures mean that theoretically 
all schbbls could be a recipient in a given year since each is cbmpared 



against Its own past p^rfdnnahce. This could cause budget problems* The ad-* 
inlhlstratlve staffs however » have found so far that their experience with the 
plan permits them to provide a fairly accurate budget estimate; they are not 
very concerned about the possibility of awards significantly exceedihg budget; 

Over the long run Houston would like to introduce new proficiency tests 
for individual courses and is beginning to work on this* This would then 
permit the plan to be converted to b% individual incentive plan for individual 

teachers 9 which they feel is not feasible now. Note that with the current 

^ - - - - - ' ■ - 

plan, teachers whose Students do not do well dh the test scdres may, heverthe-^ 

le&s^ receive an 0£P award if overall their schdol dpes well* And teachers 

whdse stufiehts dd vei^ well dh the test scdres » but who ^ school that 

ddes hdt dd well dh the tests # may hot be eligible. ThuSiy the school district 

ddes hdt label the Secdhd Hile Flan a "merit plan,*' but rather ah incentive 

plan. Heritbribus perfbmahce by ah individual teacher^ the district hotea^* 

is not necessarily rewarded because bf this grbup incentive feature* On the 

other handy the group incentive approach has the virtue of ehcburagihg 

cooperation among teacfaiers and avoids destruclEive competition* However^ thus 

far the school ^district has not specifically identified major instances of 

improvv?.d cooperation as having occurred since the plan was changed* 

The cost of the bonuses Was approximately $11 million total for the first 

two years* $7 million for the third year * $11 million for the fourth year* and 

is budgeted for $9 million in ^83-^84. (The anticipated, reduction is based oh 

ft ' • ' " 

the reduced size of individual bdhus awards fdr all but the OEP program and a 
reduced number of awardees because df the hew teacher proficiency restric*- 
tidh*) The cdst represents $700-$J.100 per teacher for bonuses in the school 
system per year* The plan also requires approximately twb full-^time clerical 
persons for the system* there is also considerable administrative time 
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reqaired in individual Schools and by central staff to track acHievemetit of 
the baseline requiremeats and to handle the extra bookkeeping items. The 
district has no estiinates of how much time or dollars thesje tasks reqtxire. 
The school district does -not do any additional testing over what it had done 

before I since it already\ had an extensive testing system anderway and would 

_ ' _ \ _ . ' . 

have cdhtihued that testing. It has not yet developed any new tests. 

Suggestio Ss to Others -^^^^ 

^ ' -^-^ \' - 

School district persoxihel had the fbllbwii^ suggestions for other 
districts: 

\. Have large enough awards, sucK as $1^000 a year^ to be able to 
motivate people. ^ 

2. Provide awards to enough people but not to all (do hot have a qiU>ta 
system). 

3. Mapt your plan to the local situation. If the school system has iib 
problem ^^th teacher attendatnce or vacancies^ don't prbvide bonuses 
fbt these conditions. That is, the school system should decide on 

areas that it wants to improve and put elements in the incentive plan 

_ -/ > - ) 

tb ehcbiirage improvements in those areas .^^ 

4. The schbbl system should be flexible and prepared to modify its p3an 
as the heed arises. ^ 

Sources 
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ii Houston independent Schbbl Districtj "The Secbnd Mile Plan," April 
i983< 

\ 

2i Elaine Say, Memo oh "Two- Year Impact Study of Secbhd^fipLle Plah^' 
April 23, 1982, Houston Independent Schbbl District, Hbustbtt, TX. 

3. Michael Mauldin, Memo on "OEP Feasibility Study June 8^ 1981i 
Houston Independent School District, Houston, TX. 
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Leslie Miller and Elaine Say^ "This Bold Incentive Pay Plan Pits 
Capitalism Against Teacher Shortages^" The-J 
Journal i Septen£er 1982 • 

5. Elaine Say and Leslie Miller^ "The Second-Mile Plan: Incentive Pay 
for Houston teachers," Phi Delta Kappan^ , December 1982 i 

6* Telephone interview vd.^t Elaine Say, Research Associate, Research and 
Evaluation Department, Houston Independent School District, December 
1983. 

7i Educational Research Services, "Merit Pay Plans for Teachers: Status 
and Descriptions," 1983, pp. 51-52. 
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King Wiiiiain Goanty Pablic Schools 

king William, Virginia ^ 

The King William County Public School District consists of one elementary 
» school (K-7) and one high School (8~12)« There are approximately 1,5X10 
students enrolled and 95 teachers. The salaries for teacher^ range from 
§12|337 to $18^162 for teachers without Master's Degrees and from $i3,537 to 
$19^362 for teacher^ with Master's Degrees. 

Description of Program ' 

; _ _ _ l'_ __ : 

' King Wiiiiam County's Taacher Incentive Program started in the 1982-83 
school year. It awards higher salaries to te^achers vhd have successfully 
demonstrated excellence in professional preparation and perfdrmance. The 
goals of the program are to attract and retain excellent teachers, reward ^ 
excellence in teaching, and thereby prbmbte superior instruction. Librarians 
and guidance cdunselbrs are also included in the plan. 

The Teacher Incentive Program has a four-step salary ladder. Movement up 
this ladder depehdg bh successful cbmpletibh of criteria for each level. 
Participatibn is vbluhtai^i teachers apply for ^each level. i 

New teachers I prbbatlbhary teachers^ or tenured teachers who have chosen 
hot to participate at a higher level of the plan start at Level I : Teacher 
Status . Teachers dh this level are paid according to the regular salary 
schedule. 

The second level. Sen i or Te ac her - , rewards teachers who- have achieved an 
average evaluation rating of 4i5 in addition to the qualifying criteria. A 
bonus of $2,006 is applied to the teacher^s base salary for the following year. 

After at least four years participation at the secdnd level, a teacher 
tiiay apply for l^veJ^^IIr Master Teacher, if the teacher holds a Master's 
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Degree or a King Wililam Couuty Prof eissidnal Growth Prdgr^ Certificate In the 
sabject area'dr grade level df asslgninent « A successful evaluation of 4.5 at 
this level rewards teachers with a $2^000 Teacher Incentive ^Program bonus plus 
a $2y00U Master Teacher Level bonus ^ as well as $1^200 for the Master's Degree 
as part of the regular salary schedule « 

_ _ __ ___ ___ _ _: __■ _1, 

To advance, to Level IV; Career teachets a teacher ;iau8t haye successfully' 
completed eight years In the Teacher Incentive Programs S addition, a 
teacher must be recommended for Career Teacher status by professional members 
of the Principal's Advisory Committees A teacher at this level wouf- 1 receive 
the $2,000 Teacher incentive Program bonus along with $4,000 awarded 'for the 
Career teacher Level, and the $i,20b Master's Degree differential built into 
the regular salary schedule • 

to remain eligible each year for the teacher lucentlve Program, a teacher 
^rnust be a full-time classroom teacher, librarian, dr guidance counselor, have 
three or fewer days* df absence that are unexcused, have achieved cdntlnulng 
contract status, have had two consecutive years of superior teacher perfdr- 
mahce evaluation from the school prlhclpal as evidenced by a Amjs average rat-^ 
Itig each year» and have made a mihimum score of 475 in the camnk>r. areas and 
475 in the subject area of the National Teacher Examlhation (or correlated 
scores on the Graduate Record Examination)^ and have had a successful perfor^ 
mahce evaluation (4.5 average rating '^oh the last rating), by ah outside team of 
evaluatbrs* 

teachers are evaluated initiaiiy and again every four years by a team of 
three experts, two members of the team are appointed by the superintendent 
and are central office staff members* the third evaluator is chosen by the 
teacher from the superintendent's list of approved evaluators, or a teacher 
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can request ah alternative evaliiatbr subject to approval by the superinten- 
dent* tJhiversity prof ^sd<r^; jiS^fe been chosen most frequently by teachers as 
their third evaiuator.^ 

Each member of the evaluation team makes at least two bbservatiohs^ ohe 
announced and one unaiixidunced* Ciassroop observations are approximate!^ 50 
minutes. Observers rate a teacher's performance on seven points: (t) in- 
structioaal methods; (2) knowledge of subject area; (3) ability to motivate 
students; (4^ classroom management; (5) student/ teacher rapport; (6) appro- 
priate studeht evaluatibh; ahd (7) demons trated student learning. Each is 
rated oh a scale of 1 (low) <6 5 (high) degree of achievement* Teachers may 
also provide the evaluatibh team with writteh dbcumehtatibh of instructional 
activities i 

A teacher can renew an award annually by bbtaihihg a 4.5 average rating 
by the principal along vrtth participatioa in some professibhal growth activity 
such as cdursework or participation in an approved thstructibhal cbhferehce. 
Teachers have the option of renewing at the same level; bbwever^ every fourth 
year the additional requirenient oiE a 4.5 average rating by an evaluation team 
must be met. The performance evaluation rating required for the salary 
supplement ±b the same ^ at all three levels. 

ft . ■ i 

Evaluatibh of the Plah 

In the first year of the Teacher incentive Plan, 10 percent of the 
teachers received awards. Fburteeh teachers had applied; however, only nine 
met the hecessairy criteria. Of the five that didh't meet the ri^quirements, 
the Assistant Superintendent felt that they had applied primarily to see what 
the program was like. 
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The Superintendent has been pleased with ihter^rater treliabili ty. At the 

_ * • * 

Inceptlbn d.f the plan it was not clear whether the rat^gs of different obser*^ 
vers would vary significantly among the group. However ^ the ratings were 
highly cdhsisteht. 

The Superinteadent noted that the absence level of teachers has declined 
noticeably^ For example, at one school^ in the 1981-82 school year, si^ 
teachers had perfect attendance, whereas during the first year of the plan, 
1982-83 ^ 17 teachers had perfect attendance. The district speculates that 
this is because teachers are trying to meet the attendance reqiiiifement with 
the intention of applying for the prbgram in the coming years. The district 
notes that although teachers say that the plan has not af A^cted their be- 
havior, the attendance statistics indicate that there has been some change. 

No evaluation has been dbhe^ such as on impacts oh student attendance or 
student learning as Indicated by test scores. At this time there is no pro- 
vision for an evaluation of : the prbgram itself. The district Is pleased with 
the results of the 1982-83 school year. The plan is bhly^in its second year; 
ther^ has nbt^ been sof f tcient time to assess the longer term oonsequences such 
as on teacher re tent ton rates* 

The administration has had only one known instance of staff dissent loti as 
a result of the plan — frra a teacher who was unhappy at not meeting the base- 
line ctiteria^ they attribute the lack of objections as stemming from the 
plan being voluntary and from the existence of clearly defined and reasonable 
requirements. « 

_ _ _ _ ♦ _ _ _ _ _ _ H» - 

The district admirlistratlbn believes that the cbinmunity suppbrts the 
prbgram and wants to reward superior teachers. The district is not worried 
abbut whethier br hot a parent will want their child in a class where a teacher 
is hbt in the program because they generally feel that the parents know when 
their child is learning a&i they already icnow who the gbbd teachers are. 



Fiicidltig *£br the program Is requested ahhually through tl^e school board to 

«,•■*■ 

the Cbuhty Board of Superlntendehts • Budgeting for teacher Ihcencive pay Is a 
separate line Item on the budget froS regular teacher salaries. In the first 
year, ^$25/600 was requested. Approx*^ stately $1,500 was used to pay for 
National teacher Examination fees and also covered the $50 6[tipends paid to 
outside evaluators to cover travel and related expenses, the system^ expects 

to add a small number of participants each year. The salary supplements will 

. __■ . _ _ _ . __ _ ^' ._ ? ._ 

be added in small increments over a number'>bf years until the program Is fully 
implen^nted. 

Sources; ^ 

1. Educatibnai Research Service'^ tnc.^ "Herit Pay Flans fbr Teachers: 
Status and Deecriptlbns^' 1983^ pp. 156-157. 

2. telephone Interview with Mrs. Claiborne R. .Winbbrzxe, Assistant 
Superintendent for Admtnistrative Servjtces^ King Willia:m Cbuhty 
Public Schobir, December i, 1983^ ^ 

3i King William County Public Schools, "Teacher incentive Program^" Rev. 
8/21/83. 
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Ladue School District 



Ladue, Missouri 
4)iatr4ct 



The Ladiiti School District consists of four elementary schools^ a juiiibr- 

, -_ . _ _ _ _ ' 

high school C7-85> and a senior high. I^adue's 250 teachers serve a student' 

population' of about 3^100*. teachers and other professional staff (incioding 

: 

some administrators) belong to the tadue Educators Associatiotx. Since 
Missouri is a "meet-and-^onf er" state, there are no negotiatioas over wages 
and salaries between the teachers' Association and the school district. 

For the 1983-84 school y^ar, entry level salaries in Ladue were $14^300 
to* $14,7004 In principle, there is no maximum salary (some teachers are 

currently earning in excess of $38,000 per year)* The average teacher 

salary in Ladue in the Fall of 1983 vas over $29it^00. — 

Description 6£ the Plan . 

Ladue's merit pay plan involves -^erf drmahce-based wage increases for in- 
dividual teachers. The plan has been in operation since 1953 and covers all 
certified personnel^ including teachers^ coifnsellors^ and librarians* Under 
the plan^ there are ^o fixed sala^ steps and no m9ximim salary levels* 
Salary increases for a given year are based entirely on the results of the 
annual performance appraisals pius^ In some years ^ an acrossrthe-board (cost- 
of-living) increase* No salary increases are %iven on the basis of a 
teacher's tenure or educational credentials* 

The perfdrmance of e^ch teacher is evaluated annually by the teacher's 
principal or (in the secondary schools) assistant principal on a scale of 0-9, 
0-ll|> or 0-13 points, depending on the salary schedule used (see below)* 
There are ho quotas oh the number of individuals who may receive a given 
rating score. Each eval^tibh point is worth a fixed number of dollars to- 
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wards the teacher's merit increases For the 1982-83 school year,' each evalua- 

. . • 

tibn point was worth $3QQ. Teacher perfdiinaace rating scores that year ranged 

from 7 to 15, corresponding to wage increases of between $2^100 and $4^500 for 

each of the district's 245 teachers. The average increase wa^ ^3^300^ or • 

about 12 percent of the average salary level in Ladue for the 1982-83 school 

— - — - r s ". 

year. Over the last several years ^ performance increases have reportedly 
averaged approximately 10 percent of a teacher's salary. ; 

Teachers in Ladue are assigned to one bf three separate salary schedules, 
depending oh experience, performance ^ and other factors. Associated with each 
schedule is a range of ^ possible ^ performance rating scores and an expectation 
as to which score corresponds to "satisfactory" teaching perfdrmance. Oil' 
schedule I (for beginning teachers), the possible performaiice ratings are 0^ 
3, 5, 7, and 9* Satisfactory perfbrmance cbrrespbhds to a rating of 5 jpoihts; 
exceptional performance coiild be rated 7 br 9. Schedule II has jpdssible 
perforinatice ratings of 0 to 11, with 7 the expected rating for satisfactory 
performance. Similarly, a third salary schedule (for teachers who have 
perfbrmed very well for several years) permits performance ratings between 0 
and 15, with 9 the exp^ct^ level for' satisfactory performancew There is no 
limi^ to how long a person can remain on a given schedule^ ' and promotion from 
one schedule to another 'can occt^r in any year* 

Prbmbtibh to a higher salary schedule is accompanied by a fixed "incen- 
tive increment" in the form of a certain number of evaluation points. A 
teacher moving from schedule I to schedule II will receive 10 evaluation 

pbihts; movement from schedule ii to schedule III results in a bonus or 12 

■ . _ ___ _ __ „ _ _ _ .<\ __ 

evaluation points. These constitute the sole basis f»^r computing the 

teacher's merit increase for that year ( the teacher^s perfbrinatice' rating scbre 

is xLot^ used to compute t^e increment when mbving tb a new salary schedule). 
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In subsquent years, however, the teacher's salary ihcremeht is again 

detennined by tiie annual perf brmahce rating. 

The perf brmahce evaluation prbceaure is highly subjective ^ although the 

process has been defined in great detail. All teachers are rated on five 

general areas: effectiveness in teaqher/^adent interactions; effectiveness 

in relationships with colleagues; effectiveness in contacts with parents, 

patrons, and community; effectiveness in ccntributijons to the total school 

program; and effectiveness in curriculum dev^eloptfiet^t ?rad prof essidrial imprdve- 

ment . Associated with each of these general :3z<;a.,* are several perf ortnance 

characteristic*:' that are used to assess teach^::^ pc:rf dnnance. Fdr ivistance,, 

"effectiveness in teacher/student intetacticr.s^' includes the fdllowihg 

perfdrmance characteristics (ambhg others): 

A. The teacher 's ihteractibhs with students reflect pre-plahhihg with 
attehtibn^tb bbth jjl^hdlvidual and group heeds. 

b. The teacher's ihteractibhs with students are suppbrted 
ihstructibhal materials. 

C. The teacher demoas crates identifiable skills in lesson presentation* 

: D. The teacher maintains a learning environment which encourages mutual 
acceptance and respect* 

For each of these characteristics » several behavioral examples' are pro- 
vided to show how the characteristic migl^ bfe applied td a given teacher's 
work. For example , the behavioral descriptions associated with characteristic 
' A above are: 

The evidence ox planning is identifiable. 

2. There is evidence of short-range planning. 

3. There is evidence of long-range planning. 

4. There is evidence of planning td accdmmddate the needs of the special 
learner. 

These characteristics and behavioral statements were developed over a 
twd-year peridd with the assistance and particlpatldn of teachers. The eval- 
uation criteria fbciis bh the teaching prdcess (as dppdsed td dutcdmes), aii 



emphasis that Ladue adtainistrators believe to be cQnsistent with the state-of- 
the-art in current research on the teaching process* They also believe that 
this focus properly places the emphasis on the cdntrollable aspects of 
teaching behavior i q 

School administr'^tdrs in Ladiie do not believe that the available measures 
of educational butcbmes (such as test scores^ can be used as valid 'indicators 

of teacher perfbrmahce at this time. For instance, student test scores in 

s 

Ladue tend to be so high that a single incorrect answer can of^ n mean a 

S 

difference of several percentiles* Because of these instabilities, testing is 

used only for diagnosis, not for performance evaluation and the awarding of 

merit pay* _ t ^ 

The primary evaluation procedure is supervisory observation, although the 

monitoring of performance targets (in the form of imprbvements recbmmended by 

the evaluator) has assumed increasing impbrtance in recent years. Ladue 

teachers have opposed the use of peer evaluatibnS: (as being tbb divisive) br 

evaliiatibns by students. Principals are given considerable flexibility in 

_ _ -^^ ^ ' 

selecting how they will evaluate and mbhitbr their staff . Teachers must be' 

bbserved at least three times per year^ each time with a pre- and pbst- 
bbservatibh conference. The supervisor's observations are written up in the 
form of a narrative (the us^bf checklists has also' been opposed by Ladue 
teachers). o 

In recent years, principals have been providing several recoinmendations 
for improvement in connection with each evaluation. These are used'^to focus 
teachers on* organizational priorities (e.g. "become more familiar with com- 
puters") and remedial needs. Achievement of the recommended imprdvements is 

not in itself sufficient to earn an outstanding performance appraisal grating. 

• '' ^ 
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All perfdrmahce evaluacibhs are cbmpleced in February and March ^ and 
' teacher salaries, including any performance rtncrements^ are announced in 
April. Teacliers may appeal the evaluation process^ to the superintendent or ^ 
the Board of educations They may not, however , ap^eai^^^^e substance of an 
evaiuacioni School administrators report receiving about -10 to 15 appeals in 
a typical year. 

The school ^.isc ict spends considerable time arid moriey trairiirig the prin- 
cipals in the process of evaluatibri. To help erisure cbrisistency ambrig the 
performance evaluatibris, the schcbl district requires central review of all 
perf ormance evaluatibns ^ the Fuperintendent , and jbint r'eviews and critiques 
of the performance esvaluatibus^ by JLhe principals as a grbup. Duritig the joint 
reviews^ principals defend their evaluatibris tb bther principals^ while the 
latter provide feedback. Schbbl administrators N^port that this has helped 
ensure a common understanding of the various ratings a^ thexr definitions. 
Periodic rotation of the school principals, and the low turn-over of princi- 
pals and other evaluators, have also reportedly enhanced the integrity of the 
evaluation process. 

The size of the merit increases earned in a given year is determined l)y 
the number of dollars that the scliddl board assigns to each evaluation pdintv 

For tUL 19o2-b3 school year, each pbint was worth $3U0 and there were no 

■ f 

acrosj-the-bdarcl increases. Fdr the 1981-82 schddl year, the bdard awarded 

-* t * - > 

$150 per pdint plus a $500 acrdss-the-bdard. increase fdr each teacher. In 

_ _ __ — 

some years, the value df a point has been as Idw as $100. However, since 

ther2 are nd qiidtas dti performance scdres (arid hence, dn the number arid size" 

I _ _ _______ 

of the awards given in any year)^ teachers do hot have tb compete against each 

other fbr a limited number of av/ards. 
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In the 1982-b3 school year, the minimam perfonnance rating received by a 
teacher in the Ladue school system was 7 points. Thus, all teachers received 
at least some increase. In prior years ^ however, some teachers have received 
hb increase at all or ah increase below the standard expected for adequate 
performance on their salary schedule.^ Teacher salaries have hot* however i 
been reduced for poor performance. Under Missouri law^ such a reductiph would 
be viewed as a demotion and would require a formal public hearing. \ 

Merit pay is budgeted by the school board as a separate line item. His- 
torical data on the distribution of rating scores is used to project an aver- 
age increase for budgetary purpdses. ,The board must also decide whether to" 
include a fixed acrbss-the-bdard increase with the merit increases i in decid- 
ing on the aiQount to. budget fdt such cos t-of-^vine increases, the board has 

.generally allocated tadst of the available funds to merit increments i For 
instance, school admiaistratbrs suggest ti;at for an average pay increase of 10 
percent, no more than percent would bt, provided as an across-the-board 
increase; the remaining 6-7 percent would be awarded oh the basis of merit. 

Teacher salaries and the amounts of any performance awards are generally 
kept ^-^nf idential in Ladue. The school board denies requests for information 
on teacher salarie s ^ although such information must be reported to (and Is 
available from) the State of Missoorii The public, however^ is reported tb'be 

^largely unaware of indiviolual teacher salaries in Ladue. Salary qotif idehtial- 
ity is also appar€|ntly the norm for teachers in Ladue's elementary schools, 
however, teachers in the junior and senior high schools reporrediy exchange 
some salary infdnnatidn; 

n l^Jote that because teachers in Ladue can earn increases even w- th 
substandard perf dripatice ^ Ladue's "merit pay" should probably be viewed as a 
combination of performance-based and cos t-bf -living increases i 



16B 



The school district makes a point of training and c attaining teachers iti 
the operation of the merit pay plan-. All prospective new teachecs are briefed 
dti the merit pay plan during their initial job interview. Orice hired^ the new 
teacher is again briefed on the perfbrmahce appraisal process and the merit' 
pay plan by the building principal ^ by the ''sponsoring teacher" assigned to 
each hew recruit^ and by the building representative to the Committee oh 
tlvaluatibn i^see below?. The building representative is also respbhsible for 
briefing all teaching staff on changes in the plan. ' 

^ ; J ■ 

A fundamental feature of tadue's merir pay plan is its emphasis on moni- 
toring and updating the program on a continuing basis. From ttte very begin- 
ning of the plan^ there has been a Committee on Evaluation charged with 
reviewing and modifying the plan as necessary^ The Committee consists of the 
-superintendent y building representatives elected by the teaching staff in each 
building, and a representative frotn the administrative staff (elected by the 
principals). This committee has conducted employee surveys and other studies 
that resulted in a number of changes to the. plan since its inception in 1953. 
Teacher participation has been high. The plan was developed in 1953 by a 

cdmmittee of six teachers and four admihistratbrs appointed by the superintend 

- - _ ■ - _ - - _- . isj'^\-i.\ 

dent . The bflgihal evaliiatibrT criteria were developed dil'^he~baBla~^f"qa '^'" 

survey of all teachers by the Cbmmittee on Evaluation, and teachers continue 

tb exercise considerable influence op. the selection of' evaliiatibn criteria for- 

the plan. The Cbmmittee oh Evaluation still meets at least bhce a mbhth tb 

discuss Ladue's nie"»"lt pay effort, and major reviews and revisibns bf the plan 

have occurred regiilarly. The fbllbwihg are some bf the changes that have b^eh 

made: 

ii Prior to the 1979-80 school year, teachers in tadue received perfor- 
mance increases of $600, $700, or $800, depending o their evalua- 
tions However, by 1978 most teachers were receiving the top 
increment of $800, and an increment of less than $800 was-Jeidely 

- ■ - ■ 170 , 
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viewe^a as a penalty i the board of education wanted the system to 
provide a wider range of perfbrmaric by increasing arid 

broadening the increments. The idea was for most teachers to; get the 
average increment, while better teachers would get a much higher 
award. A§ a result, the award system was chariged to the procedure 
currently used, in which teachers cari receive a perfbrmarice 
evaluation scpre^o'f betweeri 0 arid '9 (or 0--15,. dependirig ori their 
salary schedule)' arid a dollar value is assigned to each pbiriti 

2. Prior to the 1978--79 period, teachers were evaluated on the basis of 
three general criteria: personal characteristics, professional 
growth and development, and evidence of superior teachings The need 
to ^sifce firier distinctions iri_ order to a score of/^ 

0"i5 points led to the development of the more refined five-area*'" 
performance evaluation procedure described previously i The revised 
procedurealso incorporated the findings of recent research on 
factors that contribute to better learnings ^ 

3; Over the 19bl-82 period, thje perfortnance evaluation procedure Was 
further refined by , the development of the behavioral criteria 
described previously . 

4i A continuing issue has beeii the treatment of extracurricular activi- 
ties* Teachers in Ladue recisiye no additional pay specifically for 
undertaking such activities. This has led to cprisideraljle dissatis- 
faction, especially among secondary school teachers. After a year- 
long study, the Committee on Evaluatibri recpmmerided that there be a 
separate salary schedule for persbris supervisirig extracurricular ^ 
activities. The school board, hbwever^ rejected this apprbach arid ; 
instead directed evaluatbrs to take accburit bf extraclirricular activ- 
ities wheri ratirig teacher. perfbrmarice. Iri particular^ the ratiri|j 
must reflect both the quality of the wbrk and the time spent byjithe 
teacher bri the task (the latter factbr takes accburit bf extracurric- 
ular duties).. 



Evaluatiori of the Plan - 

Ladue's merit pay plan has undergone a number of eyaluatibris* The 
Co^itteb ori Evaluatiori tils |}eribdicaiiy reviewed the program arid surveyed 
teacher opinion. Others have examined specific aspects of the plan in detail 
(see Shaughnessy, i976)i Natriello and Cohn (1983) re'cently completed an ex- 
amination of the history and ri^olution of Ladue's merit pay plan, including 
interviews with ke;^ administrators and with 23 teachers from one elementary 
school. On the other hand, there have been no formal evaluations of the 
impacts of Ladue's merit pay plan on outcomes such as student achievement, 



171 



behavior^ and attitudes^ or oh the teaching process Itself. Most b£ the 
available inf ormatiohf on the effects of the Ladue pl^ti consists of subjective 
assessments by school administrators and teaching staff. ^ 

Scftooi administrators in tadue feel that. the merit pay* plan has been re- 
sponsible for improving, instruction (ERS 1983, pi 42)i 'Ih^s view was seconded 
by several teachers in one elementary school who reported that the' presence of 
merit pay had led them to increase their efforts and to improve instruction 
(Natrielld and Cbhh, 1983., p. 28). Such 'results are attributed to :he fact 
chat the merit pay plan and the associated evaluation system have made teach- 



ers Qore accountable tor their perf bnnahce. | School adnihiitratbrs hbte that 
teachers in Ladue are recognized and rewarded fbr dbihg things well ^ fbr com-, 
ing up with nesw approaches to old problems^ etc. Such effbrts are frequeptly 
cited in teacher performance evaluations as reasons fbr awarding increments. 

badue's merit pay plan is not credited with any special effects on 
teacher retention or the ability to recruit new teachers. School admini- 
strators itote that teaching salaries in Ladue are relatively high and that 
there is correspondingly little turnover i While administrators feel that the 

_ ".iJ o 

presence of merit pay cannot be shown to have actually attracted teachers to 
Ladue, it has also reportedly not dissuaded attractive candidatm^- Some 

teachers, moreover, feel that having me^rit pay, rather than a seSiorfty * 

___ * J__ '^__„ h-- : 

^system, has helped the school system retain its best teachers and maintain 

instructional quality (Natriello and CcJhn, 1983, pi 29) i 

Ladue schbbl administrators noted a number of other benefits from the 

_ ^/^ _ . _ 

merit pay plan. One is increased accoimtability for principals (Natriello and 

Cban^ 1983^ p. 25). Each prihcipa'l's teacher evaluations are reviewed by the 
superintendent and by the bther principals ; j they also come under close scru- 
tiny frbm the teachers. This repbrtedly encduragds the principals to pay more 



aay-to-day attentxpn to Che educational process and Co spend considerable time 
with their staff monitoring and appraising performance* Other reported bene- 
,fits include better documentation of teacher performance and greater recogni*-* 
tion of the teacher as an individuals / 

Recording to Ladue school ^administrators^ the merit pay plan has hot been 
a prbmiheht issue in labbr-^mahagemeiit relations^ with the exdeptibh of the 
question of extra pay fqr extracurricular duties (discussed previously)* (As 
noted above, teachers do hot bargain collectively over w^sges arid salaries iri ; 
Ladue . ) 

School, administrators report a high level of support for the merit pay 

plan from parents, the school board, and other members of the community. The 

_ _ _____ __ _ _^ ' 

school board has provided reliable funding for the plan since 1953 with 

increases for all who deserve them, avoiding any need for teachers to compete 

against each other for a limited number of awards. While many of Ladae's 

citizens were unaware of the plan until recently, the publicity which it is 

- ♦ ■ _ _ _._..__. J • 

currently generating is reportedly creating a favorable image for the school 

system within the cbmmuhity. 

Schbbl administrators have hot computed the total cost of Ladue ^s merit 

pay plan. Tb- primary expenditure is for incentive awards. In 1982-83, the 

_ \ ' ■■ _ ■' 

cost of awards was apprbximitely $810|0U0^ ah expense that- will cbhtihue to be 

incurred in future years siricfe the awards represent ah increase in salary 
rather than a one-time Bonus. (On the other hamdt thR.*^ e raises substitute at . 
least in p'art for across-the-board iricreases that would have been provided^ 
atherwise.) School officials note that admiriistration of the plan probably 
costs consiaerably more than administration' of a typical teacher salary sched- 
ule. It is estimated that each principal must spend about 2G hours per year 
per staff member completing the necessary ^evaluations. Program mairitenarice 
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arid staff re training require an additional 3D-40 staff-hbtars per year. And 
members of the Committee oh Evaluation meet 1-2 hours per month to review arid 
revise the misrit pay program. The plan is operated without the need for 
: special data processing, analyses, or testing, and it does not currently 
require special staff for its administration. However, when major changes 
have been implemented, it has sometimes '^'zeti necessary to cempdrarily assign 
someone to help with the plan. 

, Teacher opinion on the plan appears to be ambivalent. Host Ladue teach- 
ers appear to favor the principle of using merit pay, but significant numbers 
arie dissatisfied with sj^ecific aspects of Ladue's approach. 'For instance^ 
while 85 percent of the teachers responding to a recent survey favored a ^ 
system merit pay^ bnly 34 percent favbred the system being used by Ladue 
(bxlva^ 1983). Similar assessments have been given by schbbl administrators. 

In theii interviews of 23 teachers in one Ladue elementary schbbl ^ 
Natriellb and Cbhh (1983) fbtahd teachers repbrtihg bbth strengths and 
v/eaknesses in connection with Ladue 's merit pay plan. The weaknesses included 
the subjectivity of the evaluation process (and associated problems of favori- 
tism and inter-school inconsistency.)^ morale problfems (especially drssatis- 

__]___■__ _r_ _ . _ " _ _ _ _ _ ■ ..... _ 

faction with the treatment of extracurricular activities under the Itaerit pay 

plan) y uncertainty concerning what must be done to earn the "best" perfortsiance 

ratings and merit awards (some claimed that one had to be a coach in order to 

earn t|ie maximum of 15 points), and the absence of adequate opportunities for'- 

appealing performance evaluations (especially after they have been put in 

writing); On the other hand, th^e same teachers pointed' to a number -of 

_ •. . _ ^ .... .... 

strengths in Ladue's merit pay prbgram: the availability of monetary rewards 

for superior perf bnnahce, the encouragement of improved classroom instruction, 

the responsiveness the inerit pay system to teacher concerns, (through the 

174 * 



prbvisibh of riumerbus dppdrturiities for j#rt J cipatidti) , ariu th*? sense of 
prof essibhallsm fostered by the plan. Of the 2i elementary sclibbl teachers 
interviewed^ 22 felt that the merit pay system wbrked largely because sal&^y 
levels were kept confidential ^Natrielld arid Cbhri, p- ^7). / 

tadae school administrators report that t'he teachers' primary cbmplairit 
about the plan is the treatment of extracurricular activities (see p^ 1^^8)4 
Another is the equitability of the pay plan with respect to pay levels in 
Other school systems. For instance, teachers with masters degrees and consid- 
erable experience are reportedly able to 'aartt better salaries in other school 
systems i which reward tenure and educational attainment rather than merits 
Other cbmmbn complaints by teachers^ include the additional*^ stress of working 
under a merit pay system arid the f ruatratibh of hot khdwihg precisely what 
needs to be 3one in order tb earri a tbp ratirig (Natriellb ahd Cdhh, 1983; 
Siiva, 1983). 

Ladue administrators believe t;hat soae of these problems have occurred 
because many- teachers do not completely underi?tand the philbsophy arid prbce-- 
dares associated with the change from awarding fixed merit iricrements bf $600| 
$7uO, ^r $800 to the use of variable awards based on perfbrmance appraisal 
points, thus, 'some teachers report that they were more satisfied with Ladue's 
previous approach to merit pay, under which they received the maximum award bf 



$800. Even though the rewards are larger under the current system, these 

_ . A • - 

teachers feel that they are not performing adequately if they receive less 

than the maximum number df perfdrmance evaluation points. 

Sdme Laclue teachers repdrt that the r^^rit pay plan has created morale 

prbblems ^ causirig teachers to become money cdhscidui5 and less cooperative 

(Natriello and Cohn^ 1983* p. 32). Mbrale prbblems arid disenchantment with 

the merit pay plan are reported tb be especially interise ih the juhidr ahd 



_ ^, _ .- 

aioiT high schools. The elriinehtary school teachers interviewed by Natrieiio 
and Cbhn attributed much of that dissatisfaction to the sharing of salary 

i 

information by secondary school teachers i However, cany other theories have 
been offered to explain the reported dissatisfaction of Ladue's secondary 
school teachers, and no^single explanation see^m^ sufficient (see, for 
instance, .Natrieiio and Cbhn, 1983^ p. 33). 

On the whole i Natrieiio and Cbhh repbtt that the negative factors cited 
by the teachers they interviewed were "far outweighed by the positive factors' 
(Katriellb and Cbhh^ 1983^ p. 34). All 23 elementary ' school teachers favored 
a laerit pay plan bver other salary and evaluation approaches. This parallels 
more general assessments by local school administrators, who note that while a 
significant prdpdr^idh of Ladue's teachers have complaints about specific 
aspects of the plan, their overwhelming attitude is to change the plan but not 
to scrap iti tadue administrators see np serious possibility of discontinuing 
the plan in the* r/:- : /bie" future i 

Indeed, these administrators, as well as the teachers "interviewed by 
Natrieiio and Cohn, report that the merit pay plan has been an effective moti- 
vator for the majority of Ladue's teachitig personnel. However, it is the 
formal recognition given to excellence, rather than the monetary award, that'" 
is Relieved by many to be the inbst important mbtivatbr associated with Ladue^s 
merit pay plan ^ - , 

Ladue administrators , are uhceftaih bf^the extent tb which rUssburi^s 
isieet-arid-cbhf er labbr law has played a rble in making the merit pay plan 
feasible. In particular > they are ribt sure whether such a plan t^bulrl wbrR in 
aii adversarial setting* Nevertheless^ they hbte that if wbrkable mechanisms 
exist fbr solving prbblems between labbr and inahagemeTit ^ such a plan would 
prbbably have a chance anywhere. 
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senior high Schools. The elementary school teachers interviewed by Natrielib 

and Cohn attributed much of 'c;; dissatisfaction to the sharing of salary 

. ^ : L 

information by s€con::^ary school teachers i However, many other "theories have 

been offered to explain the reported dissatisfaction of Ladue'^s secondary 

school teache^rsy and no single explanation seems sufficient (see» for 

instance, Natriellb and Cohh^ 1983j p. 33). 

Oh the whole » Natriellb and Cohn repbrt that the negative factbrs cited 

by the teachers they interviewed were "far butweighed by the 'pbsitive factors" 

* (iNatriellb and Cbhn^ 1983, p. 34). All 23 elementary schbbl teachers favbred 
a merit pay plan bver bther salary and evaluation apprbaches. This parallels 
more general assessments by local school adminis tratbrs , whb hbte , that' while a 
significant prbpbrtibh bf Ladue's teachers have complaints about specific 
aspects of the plan^ their; overwhelming attitude is to change the plan but not 

. to scrap it. Ladue administrators see no serious possibility of discontinuing 
the plan in the fortseeable futures " * 

Indeed, these administrators, as well as the teachers interviewed by 
Natrifelld and Cohn, report that the tnerit pay plan has been an effective tnoti- 

vatdr for the majority of Ladue's teaching persoiinel. < jjowevex^- i t i s, t he 1— 

fonnal recognition given to excellence^ rather tJtian the monetary award, that 
is believed by many to be t associated with Ladue's 

merit pay plaii. 

Ladue administrators are uncertain bf the extent to which Missouri's 
meet-^and-cohf er labor le^w has played a role in making the merit pay plan 
feasible. In particular ^ they are hot sure whether such a plan would wbrR in 
an adversarial setting. Nevertheless^ the^y note chat i^f •workable- mechahi sms 

____ , _ ■_ ___ 4, 

exist for sblvihg problems between labbr and management,, such* a plan wbuld 

_ „ • 

probably have a chance anywhere. 
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Lebanon Public Schools 
Lebanon > Connecticut: 

The School District 

Lebanbd Public Schools consist of two schools (ah elementary and a high 
schbdl)'» 80 teachers and apprbxlmately 1^200 students. 

«i 

Description of the Flan 

Lebanon's merit pay plan was contracted with the teachers'* union for the 
thrrte-year period froi? 1977 to 1979. 

The merit incentive evaluation plan In Lebanon Involved bonures fjr a 
peacher's progress In toeetlng objectives set at the beginning of the school 
year and for satisfactorily fulfilling the requirement's of the, teec.hsir's job 
'escriptlbn. The. plan begin ±n September 1977 and terminated Iti the third 
yea abbut May 198D* Each t' er \^as required to identify bne perif brniance 
oujecnive fbr students in te .s subject matter arid bne process dbject.-^./e. 
The methods by which achievement of thc^v objectives wou j bfi measured were 
also to be specified, the process objectives indicated the procedures that 

_ H • ■ 

the teacher would use use to meet t ie perf 'rmance objectives. Tho second part 

- 

_ _ _ __ ___ ' 

of the evaluation process involved use of the tear.ier's job descrlptlo i as a 
checklist tc evaluate the tx :*ch^'s p€*rf ormance in terms of four goals and 20 
aties 36 prescribe^ by th^* school diotirict. 

At t*>e beginning of the school year, teachers submitted their objectives 
to the principal for his concurrences Five times during the year tea^^hers 
submitted to the principal an interim report indic'atirg the amount of 
progress. The report cdiild include requested revisions to the original 

bb jectlvcs. The principal twice during the year completed job description 

- - - . - 



pri- viipal based oh their performance oh the performahce objective (lip to 200 
points), on the process objective ^200 -points) ^ and oh the job description 
facing (560 points )i At the end oi May of each year^ the principals submitted 
to the superintendent and school board their recommendations for merit inceh- 
tive recipients based- on the extent to which individuai ceachers met their 
objectives and fulfilled the requirements of their job description. (However ^ 
the reiatidhship betweeh the numerical rating and the size of the bonus was 
not clear.) The merit pay was distributed in Jure. 

Teachers at the elemf»ntary and high school levels were ranked i^eparately 
. to encourage consistency within each school and in light of differences in 
teachers' responsibilities between elemehtary ahd secbhdary levels. Those 
teachers thar achievep rankings higher thah the expected level of performance 
were sltgibie to receive bonus awo^rds. In the first year of the plan (77^78) * 
rj^'QUses ranging from $100 to $500 were paid to 55 of 78, persons (about 70%) ^ 
with an average bonus of about $2 "0. The bonus size went up In the last two 
years to $900 maximum for the third year, however^ 

The plan also covered other staff such as counseiars, school p/iycholo- 
gistSi and library specialists, as well as teachers. Different evfiuatioh 
forms^ relevant to each group's job description, were useidi / 

Evalv jition of Plah : 

the Lebanon plan was ihtrdduced by the school s .iperintendent and wa? 

initially supported by the teachers and the community. But at the develop- 

« _ . _ _ _ _ _ . _ 

mental stage, even prior to its actual implemeptatibh, tiie pl'ih met with 

resistance from a newly elected union leadership that did not support the plan 

ratified by its predecessors. The hew leadership^ elecled in part because of 

its opposition to the merit pay plan, did hot permit formal participation by 



participation* As Che first year progressed » tension Increased within thk 
schools. A primary problem was infighting amoag teachers, which Incliidecl 
jeaJousy and hoarding of cnaterials^ creating a divided staff. 

Another prdblem identified by the superintendent was the short time 
period available to develop 'the \.)lan--£rop April 1977 when the contract with 
the teachers^ association was signed antil September 1977 when the pian went 
into effects This was aggravated by the lack of staff response to the super- 
tr.tendent'8 rjequest for part icipation in the development of the plan s details 
(due in part to the changed attitude of the new association officials)* 

Another major problein arose after the aWards were made shortly before the 
close of the first year. Although th^ contract cdntalaed a fo\rtxial agreement 
calling for cdaf Identlality as to who received the awards, InfortnatlOG leaked 
out^ appa;..^atly fro^ son^ teachers. Public disclosure of this infdriDatidn 
discouraged some teachers^ Community problems arose it this time, and parents 
began to request that their children be assigned to certain teachers. 

A critical concern oi the C -^chers was the la k of objectivity in the 
evaluation piocess. TeacJiers felt that the possibility of favoritism left 
some teachers at sn aciv^atage tQ recel/e awards. Elementary teachers and the 
elementary school principal were parti 'iular^O^ negative. High scHbbl personnel 
:were less negative^ perhaps because they tended to ba mpre recent hires. 
Also, elementary school teachers felt that since they were laying the educa- 
tioa£>l fouadaLion for the students, high school teachers h:., predisposi-:, 
tldti for success, and as s 'ch, the e] ^^uentary teachers shouict receive a larger 
award thati high schdoru teachers i 

The district also found that the lextra pap^i.-wdrk, includitig the various 
interim reports ^ became k significant burdeii. 
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Although the superintendent felt that the plan had a favorable effect on 
student achievement^ the plan was short-lived • ' He believed there was a 
noticeable improvement in student . learning^ e.g» increased scores on tests and 
greater participation of students in extra-curricular activities.^ However, no 
formal evaia^tion of the plan was conducted. As dissentibh among sta^ff 
raptabers grew, reportedly it became "very demoralizing" and eventually led to 
the plan's demise in 1980 at the end of the negotiated contract term. 
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Penii M*iiibr School District 
Millersville ^ Penhsylvahia 

The School District 

The Pehn Manor Sb;hooi District c3nsists of ^^gtit schools — ^ix elementary 
schools, one middle school (7-^)*^ and a high school. - The 'total enrollment is 
curre|Qtiy 4,166 pupils • Non-supervisory personnel belong to the Penn Manor 
Education Association, an affiliate of the NEA. They bargain collectively 
over wages and salaries i At present there ate 220 persons (teachers, counse- 
lors^^ nurses, librarians, and other ndn-supervisdry staff) in the Wrgaining 
unit* Teacher salaries start at $12^000^er yea|: ^and can go as high as 
$28,500 per year for teachers with a nj^sters degree and is years of experi- 
ence- The average ^lary level is $19*^000 per year. 

Descriptibh of the Plan 

Penh Manor's merit pay plan provides cash bonuses for butstahdihg 
teachers and bth^r members r' » bargaining unit based oh supervisor observa* 
on and ratings. ' >ian was formally begun tn November 1982 and cur- v 
rently in its second year. It was included ar part of a three-year contract 
with the Pen:£ Manor EducaCioc Association signed iTi August 1;98^. The. agree- 
ment stipulates that the Board of Education provide - $26^666 in merit pay for 
e?^ch year of the three-year contract. The merit pay provision ts in addition 
Kz across-the-board salary increa8.es. The plan covers all persons in the 
bargaining unit- 'tidn-supervisdry teaching staff, gui.^ance counselor , nurses, 
librarians^ ^ic. Particlpatibh in the merit pay plan is /dluhtary. ^ 

The same amount is awarder-, to all qualified persons. The size of the 
award is determined by dividing the $20^000 award pool,: f given year (ijIu^^ 

any additional reward mbhcy ? .wl rized by the schobl bbaru> by the pjn:L»L.r o!: 
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(about 11 percent of the covered persbhnel) earned awards of $1^000 each aft^r 
tH^ School Board provided ah extra $5^000. The additional funds were.cbm^ 
initted to ensure that the awards would be at least $1^000.^ (This amount: was 
felt by the School SoErd^ Chairman to be necessary if the program was to be an 

: : 

effective motivator.) The first-year bonuses were pair^ in Hay, before the end^ 
of the school year, ^e 25 recipients included 23 teachers and two counselors 
(but no nurses or Jibrarianr,) . ; ^' 

Teacher Evaluation aad Selection of Reward Recipients . The procedures 
for ev^luatiri^ teachers and selecting recipients have been different in each 
year of cues program. The first-year procedures "are deocribed below j second- 
year procedures are covered later-, 

During the first year^ merit pay candidates rxad to be hbmin^.ted for the. 
award. Anyone could submit a nominatibn. Th^ nc .ainati6r?5 ^-f^ri* mcde anony- 
mously and had to be received by December 23 ^ 1982c / itrlbh proce- 
dure, ihvbiving the following eight criteria ^ was u ed ...dhatluf^ aad 
S'Slecting the merit pay recipients: 

1. QOf lity of planned instruction 

2i Effective irf^iementation of planned instruction 

3. Cdtnmand of subject area. Including depth of understanding an^ 
knowledge of current; trends 

4. Effort and effectiveness in mbtivatlng students 

_ _____ _ __ 

5. Accuracy and effectiveness of cbmmunicatidns 

5. Resourcefulness and adaptability ^jtc change 

7. Involvement In total schobl prbgram 

8. Rapport with student- staffs parents ^ and cbmmtinity 
The nbminatioh had tb indicate qtialitatively that the candidate excelled 

with regard to each criceribn. Il the^ first year^ 158 of the 222 eligible 
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All ricniinees were then sent p ' t- -wv- iiiatr..on f^ nn bri which they w«ere 
asicad to assess aad document the performance with regard to each 
criteriorN A fi\e-poiat scale was emplcy«4; »jcwever, little vas done co 
define or c erwise anchor the criteria. Oiie-hutidred and thit?ty-one teachers 
^ completed the self --evaluation, questiormair a and became finalises- 

Each finalist was theti evaluated a t'^am of three supervisors, using the 
same criteria and 5 point scale. Different teaTDS were used for different 
schools* In general » the teams consisted of the tea'^her's Iminediate 
supervisor and two other persons. (For elementary school teachers ^ the 
additional persons were principals from other elementary schbblsj fbr middle 
and high school teachers » the committee was usually cbmpbsed 6£ the teacher ^s 
department chairman, i.r\ assistant principal^ anr -- schbbl principal.) 

Each member of an evaluating team was req j observe the teacher in 

the classroom for at least 45 m±^at^^^^^J6ne period br^ for elementary schbisl 
teachers, one subject aref;). These oblservations were conducted separately by 
the ::mefflbers of the team^ In some cases they were announced, in some cases 

unanno^mced; Each three-person committee * R*i 18-20 candidates to evaluates 

_■ _ _ " 

Afier all evaluations were completed, each committee had to coasolidate the 

■ - - - - - - ■ - - -' 

separate numerical assessments that the committee tnetzibers had prepared for 

each candidate ai^ci rank order r.he candidates; All assessments and rankings ^ 
were completed by April 15, 1983. 

The vavibus cbmnJLttee rankings were t ieri synthesized by a five-*persdn 
steering cbnimiti-^^i with the help bf the varibus schbol principals. (The 
cbmmittee cbhsisted bi the assistant to ti.-^ «-.up^ririterideiit , the elementary 
school coordiiaator , the middle school principal ^ the high schbbl principal, 
[ and a department chairman.) The steering cbmmitte^ asked each schbbl princi- 
pal to rank the candidates from their schbbl on the basis cf ths evaluation 
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committee assessments. 'Hie steering committee then winnowed down the number 
of candidates until the number remainihg was small enbiigh to provide a signi- 
ficant award foi each recipient; (given the $20^000 award pool). The committee 
was una>le to reduce the list of candidates to fewer than 25 teachers. Be- 

- cause they felt that it was important for each recipient to rece^ive .a^ least 
$lyOOOy the steering committee asked the Board of Education to provide the 

• additional $5^000 needed to permit awards of that size. 

The names of the 25 recipients of the $1,000 bonus were announced by the 

. Board. - They were also' published by /the local newspapers. - a • _ 

• r - > V 

The performance evaluation process was admittedly very subjective. How- 
ever, schooi administrators note that the committee evaluations were generally 

' ** _ ' _ _ _ 

accepted by the teachers. The schobl/district has been , regularly observing 

and '•ating teachers (as satisfactory vs* unsatisfactory) for 15 years. Ten- 
ured teanhers have to be observed at least twi^e is a givfeh year; hew teachers 
are observed twice ^ach luartcing period (eight times pet year). Since the 

school district is sm^ll, teachers kn* the evciuatorj. weii^ and problems 

\- , 

\ _ •_ 

concerning the qualifications of the evaluators and the aubjectiv^y of the 

assessments have hot ariseti. ■ 

No effort was made^tb ailocate tab niimbar of iwards between the .schools 
or levels. Nine awards went to high school, teachei s, five to middle schooi 
teachers^ and lleveh to elementary sc^hddl tsachers. Administrators admit to 
some problems with cbmparabillL'^ across schools. For in«tance> some princi- 
pais were regarded as more lenient in their^ ratings than others. T9 alleviate 
this problem^ the individuals cbmpr:lsing the evaluation team tor a given can- 
didate were chose i i^o provide a balance between harsh ^nd more iehicnt eval - 
uatori5> Ir. tSe cape of the^h::^gh s^o6i and iriddde school^ ^oiapatability in 
evaluate , lis w;iB snh- aced by hav j ;tg the p-:ihciral rev-^-iw cnn evaluations for 
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No fbinnai appeal process was specified for the first year of ^the program. 
For the second year of the program, principals and evaluators have been in- 
formed that a candidate has the right to involve the superintendent or the 
assistant to the superintendent -if there is a difference of op: b^^tween 
the evaluators and the candidate with regard to the latter's '^otmancei 

the merit pay program was drafted over a period of several months by the 
assistant to the superintendent, L. . oientary school .cddrdi i^ ror , the liiddle 
school principal, and the high sch » . j.ncipal. While they considered vari- 
ous mechanisms for getting teacher participation, they decided that this would 
lead to excessive delays. The program was implemented in Nbvem" "^r 1982, and 
hbmihatibhs fbr^the first round bf awards were received by DecGmber 23, 1982. 

There was hb fbtmal particlpatibh by the Teacher's Assbciaf^bh. • The lat-- 
ter maihtaihecf. a formal hahds--6ff appc3ach^ although it frequently urged 
teachers not to return the various que^ tionhaires and rating f biruis needed to 
administer the program. No pilot testing wa« done. The Board of Education 
accepted without change the plan developed by the four-person committee* 
After the first year^ the Admin Is trs Clou ^siced the teaching staff for sugges- 
tions for improving thci program. Very few suggestions were received, except 
with regard to the uomln^tiog inrocess*^ However, there were some complaints 
that teachers did not understand the program* ^ 

No specii^.l training has been reculred, and no additional administrative 
st^f f* wete- adaed to run the program. Once Lhe plan was approved, the princi- 
pals wtit with their department chairmen and explained the program. The pro- 
^ram \ as explaihie^ to them again auring the Foiii^ier session. The principals 
'?,lsb expl^iip^d the prbgram tc 411 fciculty mtmbers during" regular faculty 
meetings^ 



The State Department of • Education had no foniai or informal role in the 
Penh Tftanor merit pay piani indeed, the Department of Education has reportedly 
hiH d hahds--off attitude with regard to merit payi 

Chan ges' fefter the Fir s t Year of the Plan . During the sunuL'^i ci .:^83| 
he Steering Committee sought suggestions from teachers cohcerhihg ways to 
improve the plan. Although the uhidh urged teachers not to respond to the 
request, a few suggestions were received. One issue was the publication of 

— - - IW 

the names of the award recipients. School administrators, however, feel that 
since information bh the salaries of pubiic employees is a popciar topic of 
local newspapers, the ideht-^.Jicarion and publication of the recipients of the 
merit awards is inevitable* tliere were also complaints about the i'^ominating 
process. Nominations could be mads by a friiend or by oneself, and many- 
teachers complainefl that .they did not unuerstand why they were not considered 
for a given award; 

' For the second ye^r (1983-84), the following 'changes were made in the 
merit pay plan? 

Every employcie is to be e.aluated (through sujparvisbr bbservatioos) * 
at lease V:ro times before February i. This is expected to defuse qu^stibas 
abo'it the adequacy bf superyisbr assjessuiehts bi the candidates. It vtIII also 
/ prd\Kde added docuicehtatioh 6h employee perfor^-ariie. 

• Tka self-nbminatlbh pxibcess has been dropped. Any employee will je 

^l^g Tile for merit p^ y awards if wb sat tsfacrofy evaiti?tibns are on tiJf.ei 

3. Ihe evaluation process fins been revised i The form licS been altered 
to require ratings of several specific characz:si? ^nics for each of the eight 
crii'eiia iTsted earlier^ A total cf 28. such itetas .is provided. The teacher 

is |to be rated on a scale of 1 (marginal perf ortr^-tnc'i) to '> (dxstingu-'.shed or 

- - 4 .- - •• - - - - o " - " _ - - - _ - 

o^it p tanking, peifcrmatice) on aach of these j terns (a five^-pbint sc?le .as lisaK . 
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Each teacher is supposed to complete the checklist for himself. This 



self -assessment is not, however, submitted to the teacher's supervisor. The 
teacher is also required to describe his teaching goals on the eykluatibti 
form, what has been done to meet them^ acHievemehts during prior years^ ^tc. 

In additibn^ the teacher must indicate whether or not he wants to be consid- 

- __ ^ ^ . . _ 1 

ered for the merit pay bonus. The description of g6ais and achievements and 

Che statement on merle pay' participation are to be completed by early Deceoiber 

and submitted to the teacher's supervisors • . 

Each teacher wishing to be considered for me xt pay ^is then evaluated by 

two persons- -the tearchei's inizaediate supervisor . :td the schoos^ principal — 

using the checklists These assessments are co^. . ::ted independently, brie of - 

the two. evaloators m"st visit the classroom ag-^ a and assess the teacher's. 

. - • -- - _ - p - * _ - - - _ - _ _ _ .- 

p*i:ri v^rtuance. The two evaluatdrs meet and develop 4 cdnsdlidated ^et of 

_ , \ _ _ _____ 

iaciugsi The average rating score from this cdnsblldated checklist forms the . 

! :i-:lc> Cor tUe av cds* " . ' ' - 

*- .or*: jn^>-t:;.r; their "^inal recommendations) the two evaluatdrs are to meet 

with each teacheir to discuss the ratings they have come up with* The teacher 

can at that time refer to his seliC assessment of. the items bh the checklist 

aiiJ discuss ^^vy discrepati'les between hiri* r -.tings md those of* the evaluatbrs* 

If there are uaresblvabie disagreements^ the superintendent or assistant^ 

' ^- 1 ■.. ■ 

supariac.'^ndent can be involved* . • • 

K valuatibn of th^ Pi aru " - 

No formal evaluation of the plan hi. , been condfxcted. The evlucation* Asso- 
ciation claims that jie plan has had negative eff (jetton staff members, while 
Fcnooc districr, nianegf^ment disagrees. The f ollot^lng -sub jec -ive assessments 
have oeen puviti^d bj^ schddl admltiistxauors. ^ 



School administrate ^port several impacts on teacher behavior that are 
apparently partly relaCed r.d the merit plan. Responsible students, when asked 
informally by the principal, have reported that teachers appear' to be trying 
harder and to be better organized thif> year than in years prior to the merit 
pay plan. - School principals report that lesson plans (which must be turned 
weekly) reflect considerably more thought and represent a marked inip|rdVement 
from previous years. The principals .also repr rt more innovation by teachers 

this, year; including new science and "niKvh ip^irs- 

_ '__ _- __ ,f 

Another .':fflportant benefit of . tan cited by school officials is in- 
creased willingness and interest by teachers in volunteering for extra assign- 
ments. They report that teachers who never before had offered to help with 
extra-curricular activities aud other voluntary tasks are now" ^^ekiug them. 

School officials note that the district has haJ tsore- national mirit semi- 

: . . 

fii\e;lists. this past year than ever before. The officials, howev^^^, point out 

that this could be a cbihcidehce and not a product of the merit pay programs 

/ There:, have been a fe%^ complaints from students that certain teacher. lo 

should have reeetved awards did hot get LHem. But no complaints have bee-^- 

received that teacRe.rs who had received awards should not have gotf-ter uhem; 

School off icials report few negative reactions; to the merit pay plan. ^ 

Although union officials feel that S^rale has suffered^ school prim^ipals feel 

that morale has never oeeit higher and that they had t- • school 

opening ever. They feel that: teacher attit ides towardi- ' f '^^m are , better 

than expected and that there Ixais Ven relaciveiy little oppositiou to the 

plan, although there are some bugs to be worked outi 

No change has been dbser^^ed in turn-over rates, which are generally low > 

(less than 5' percent). School administratox-s , howeve::, attribute this to the 

shortage of jobs for teachers. 



All candidate teactiers are briefed .pn the oerit pay plan and asked how 
they feel about iti All have reportedly said that such a program is "great" 
aad^that^ they would; like to be paid on the basis of what they produce,. One 
^ rectuit reportedly applied for, a job becaiise of the merit pay plan. 

School admlriistratbrs feel that the leadership of; the Teachers Associa- 
tion h^.5 become negative towards the plah^ even ti '^.i ,,h they signed the thres- 
year contract which contains the program. No form-l grievances have beei 
filed over the merit pay plan* 

No complaints about the plan have been received from parents, and the 
plan has attracted little op'posit ion from the gef^eral public? Of comments 
received, approximately two-thirds were supportive of the plan, one-^third 
opposed. 

The so::e out-of-pocket cose to the school district thus far has been the 
§25,000 pai in incentive awards — about vll4 per teacher in the district. 

School administrators estimate, however, that at least '$50,000 worth of time 

< -' - _ _ _ _ * _ 

was spent in the development and admlnistratibh of the program. No detailed 

accoiiiitihg, however', ht^i bobn'uhdertaken. | . . ^ 

Overall^ school; officials believe that the program has been successful^ : 
surprisihg even some skeptibal administrators. They feel that the program has 
definitely served to motivate teachers to try harder. The motivation report- 
edly arises not so much from the cash awards as from the recognition now pro- 
vided to outstnnding teachers. Prior to this plan, teachers were evaiuaced ; 
merely as satisfactory or unsatisfactory i Thetserit pay plan provides a me^ns 
for identifying a teacher as being outstanding. T iis has reportedly stimula- 
ted good - and even weaker - teachers to try harder. ; 

It . i-s the current intention of the steering comiiiittee to provide more 
awards during the cecond ye.?r of the plan. Since the award pdbl^is fixed,* ; 



this will mean redaciSi the size of the bbtius given t8 each Recipient. It is 
felt that the bbnuses coaid be paied back, to $750 apiece and stiii be mbtiva- 
tibrially effective. (If §25,660 is available, this would permit about 33 
awards and cover 15 percent, raihki than 11 percent, of the teache^iO The 
administration feels that it would be desirable to ultimately provide awards 
for perhaps 25 percent of the teachers i 

Suggesttons to Others 

Penn Manor school officials suggest that administrators interested in in- 
stituting merit pay start with an acceptable salary scale for all teachers and 
add a merit pay pool to it, as Penn Manor has done. The program should not be 
undertaken if the union views the existing salary levels as unacceptable. 



Sources: 
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Round _Valley School District 
Cbveib » California 

the School District 

Round Valley has two schools (K-8 and 9-12) plus a cdntinuatidn high 
school located near the high school i It has approxiinately 460 pupils and 23 
teachers. It is a somewhat isolated school district in Northern Calif brhia 
with "self-motivated, individualistic" teachers. The teachers are represented 
by the California Teachers Association, an affiliate of NEA. 

Description of the Plan 

The incentive pay plan provides bonuses of up to $2866 per year per 
teacher i^in addition to any across-the-board cost-of-living increases on the 
salary schedule). The amount of the award depands on how many of a maximum of 
10 points a teacher earns. The 10 points are divided into three parts: up to 
4 points are awarded oh the basis of teacher initiative in developing a new 
project or activity; up to 2-1/2 points are awarded for projects involving 
teacher cooperation with other teachers to achieve a common objective; and 
3-1/2 points are awarded based on an evaluation of the teacher by the princi- 
pal. The plan began in 1980-81; 1983-84 is its fourth year. There is a pre- 
set bonus schedule that specifies the size of payment for each point awarded 
(approximately $286 per point). 

Participation is voluntary; however, almost all teachers have partici- 
pated. In 1982-83 one teacher who objected to the plan on principle did hot 
participate. ' 

In 1982-83 the average award was $1700 with a high of $2466* Ail but one 
of the teachers received at least some amounts 

Because 25 percent of the award requires joint activity by more than one 
teacher, the plan is both ah individual and group incentive programs For the 
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teacher cooperation projects ^ the staff is divided into four groups: K-3, 
4-6, 7-8, and 9-12. Teachers can develop projects v/ith one or more teachers 
in their grdupi 

♦ ____ 

The amounts and winners of the awards are kept strictly confidential to 

eliminate the necessary tensions arid possible embarrassment before students^ 

fellow staff members^ and parents. 

The awards are determined at the end of the school year and distributed 

soon after. 

Mi teachers can receive the maximum bonus in any year; there are no 
quotas. The school-board puts into the budget $2800 (the maximum award 
amount) for each teacher in the school district. The allocation between the 
two schools, thus, depends solely on the number of teachers in each school. 

Currently only teachers are covered by the incentive program, but the 
school district is planning to adopt a management bonus covering its six 
administrative personnel, excluding the superintendent. 

The bonus plan is covered by the teacher's association three-year 
contract. The current contract is the second one that includes the plan. 

JTeacher Evaluation 

Both the "teacher initiative" and "teacher cooperation" activities are 
evaluated by a committee of four people, consisting of the school's principal ^ 
a teacher chosen by the unioiii and two members of the school board. The ele- 
mentary and high schools have separate committees, but the two board members 
are the same, providing greater comparability of the evaluations. (The Board 
of Education can be so heavily involved because of the small size of the 
district.) 

In the fail the committees set the maximum number of points that can be 
awarded each teacher depending on the cbiSittee's judgment of the value of the 
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individual projects* At that tlme^ the teacher and cbmmlttee determine what 
criteria the specific project should be evaluated against. (Criteria could 
Include observations of completed projects^ judgments about their success^ and 
test score evidence • ) 

« 

In the Spring, the committees meet with the teachers to review the re- 
sults of the activities and determine the actual number of points to be 
awarded. Thus far, test scores have not often been used to evaluate projects; 

The actual scores for both the teacher initiative and teacher cddperatidn 
projects are determined by vote of the committee. There has been a consider- 
able amduht of consensus among committee members. There is no formal appeal 
process^ but teachers can appeal to the com * -tee. There was one appeal in 
82-83 which was reviewed but turned down by the cbmmittee . 

For the evaiuatlbh by the principal (cbmprlslhg 35 percent bf the 
points), two approaches are used. Firsts the principal makes three classrbbm 
observations over the year (the cbntract with the assbclatlbh calls for a 
mxnimuffl of two observations) i Second, each teacher in the fall develops with 
the principal a plan for evaluation. (Such a plan is required by state law^ 
but the state requires this every other year whereas the school district does 
this every year.) The principal in the Spring judges the extent to which the 
teacher has achieved the plan. The principal's rating could be based primar- 
ily dii judgmental factors or, depending upon the nature x)f the plan, on more 
objective criteria such as test scdres. 

The superihtendeht is ndt involved in determination of who receives 
awards br their size. 

Evalaation bf the Plan 

No formal or informal evaiuatioh has been undertaken bf the plan. Nb 
evidence has been collected bn, such elements as changes in student test 
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scores j teacher reteritlbn arid absenteeism rates, or oh the viewpoints of the 
community i parents ^ pupils, or teachers. The superintendent (who came to the 
district in May 1983) suspects that there has not been a substantial impact dh 
student ? earning--not because of the plan itself, but because of the lack of 
any focus by specific objectivei. He hopes in the future to obtain more spe- 
cific guidance from the school board on specific bbaectives such as to improve 
student reading or vocational education and then to encourage these as a focus 
of teacher projects; 

The superintendent did feel, however, that the piah was definitely 
encouraging teachers to do constructive things. Teacher projects haVe 
included §uch activities as teaching library skills to junior high school 
students during a teacher's preparation period (the project of a first grade 
teacher with a master's degree in library science), offering an elective 
course on advanced mathematics to eighth graders (a project of a second grade 
teache?), and working together to establish writing, health, and physical 
education programs. Some teachers have earned merit points for cooperative 
efforts in orgahizihg contests and extracurricular activities. 

The teachers have been cooperative, with diay a small number of excep- 
tions. The union has been cooperative; as noted, basic elements of the plan 
are included in the teacher's contract. 

The primary element of cost for the program is for the grant awards 
(which averaged $1700 per teacher in 1982-83). A. small amount of added 
clerical work is required for processing the paperwork, and committee members 
spend substantial time in meeting with teachers and assessing projects in the 
Fail and Spring. 

is of thf brmatibh 
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Selling Oklahoma Public Schools 
Selling, bklahotna 

the School District 

Selling is a very small district with an enrollment of approximately 556 
pupils. It has one facility covering K-12 and approximately 44 teachers. 

For the t983-84 schooi year, the beginning salary for a teacher with a 
bachelor's degree and no experience was approximately $15^000; $20|200 for 
teachers with 15 years of experience and a bacneior's degree; and $21^500 for 
teachers with 15 years experience and a master's degree. 

Description of the Plan 

Awards are based solely on the results of test scores on student achieve- 
ment. The plan began in the 1979-80 school year and was in its fifth yeat; of 
operation in 1983-84. 

The plan uses both group incentives and individual bonuses. Participat- 
ing elementary school teachers (Grades 1-6) receive a $500 bonus if reading 
scores in their school reach the targeted increase. Each teacher whose class 
reading scores reach the target receives an additional bonus of $250, and each* 
teacher whose class math scores reach the targeted increase receives an addi- 
tibiial $250. thus, participating elementary school teachers can earn a 
maximum bonus of $1,000. 

All participating secondary school teachers (Grades 7-12) receive a $300 
bonus if the reading scores in their school reach the targeted increase. In 
addition, each participating secondary teacher earns a $140 bonus for each of 
the five classes in which their students reach the goal for that subject or 
skill area. Thus^ secondary school teachers can also earn a maximum bonus of 
$1,000. 



199 



S ^ 2 



A iarge proportion of the teachers have received bbhuses during each year 
of the plan; In 1983, 38 of the 43 teachers (88 percent) earned merit pay 
awards, varying from $300 to $1,000; The average award was $853 (for a total 
of §32*440); The school bonus has been earned in each of the first four years 
by both the elemehtairy and secondary school; thus all participating teachers 
received the basic school bonus ($500 in the elementary school and $300 in 
Grades 7-12). The superintendent estimated that approximately 70 percent of 
the teachers have received the maximum of $1*000* indicating that their 
classes had reached the targeted test score imprbvemeht • 

Class test score gains are determined by cdhvertihg the standardized test 

scores to "normal curve equivalency" poi^its (NCE), These points are averaged 

for the students in the class* the school gain is determined by averaging the 

NGE scores for all students in the school. The target for the increase in the 

NCE score is set by the district administratioui it initially was set at two 

points, but in the third year of the program, a committee of teachers and the 

principal recommended raising the required test score gain to six points above 

normally expected gains (two points had proved to be too easy a target). 

The school system gives reading and math tests each year, both in the 

/ ^ _ 

fall and spring. The Comprehensive test of Basic Skills has been used, and 

hew tests have been developed, particularly for several high school subjects i 
These have been developed by the teachers with a consultant from Central State 
University. The Oklahoma University scoring service has scored the standard- 
ized tests. Teachers score the other tests with principals checking them; 
the district uses a micro-cbmputer to calcuate the change scores, each class' 
and school's averages, and the amount of bonus each teacher will receive; The 
pre- and post-test scores are keyed into the computer. 

20Q 
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The bonuses are provided in addition to across-the-board increases ^ which 
have been given each year to all teachers. The state has annually mandated 
specific across-the-board increases except for 1983-84, but the school 
districit itself gave everyone ah increase of $750. 

The bonuses are given out at the end of June or early July, as soon as 
possible ifter the awards have been determined.' The year-^end tests are given 
in April. The two-month cfelay is primariy for obtaining the test scores from 
the scoring service. 

The plan is vdluritary: a teacher wanting to participate sighs a separate 
contract in October acknowledging that the bonus is hot to be considered 
salary and will hot be carried forward to the next year^ With only fare 
exceptions^ all have volunteered. 

Ajdmihistratbrs are hot included in the plan, nor are band and music 
teachers, who receive extra pay for those act ivi ties i 

No attempt has been made to consider differences in student characteris- 
tics among classes of over time that might make it harder or easier to achieve 
the target test scores^ 

the plan was developed solely by members of the administration, with 
technical assistance from the State Department of Education. Teachers did not 
participate, the district, however, uses a teacher advisory council, with two 
teachers elected from the elementary school and two from the secondary schodl. 
It meets about twice a year and suggests mbdif icatibhs to the program. The 
teachers are members of an NEA affiliate, but the association has not formally 
pafticipated in the development, or implementation of the plan^ 

Teachers can appeal to the superintendent, but thus far «-here have been 
very few appeals. 
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The district, as required by state law^ has ah annual teacher evaluation 
program, with teachers rated annually by the principal. The merit pay plan Is 
Intentionally kept completely separate from the teacher evaluatlbh process, 
if a teacher's classes did not achieve the targeted gains required under the 
merit plan, the teacher would not be penalized unless the principal otherwise 
found problems in the teacher's performance. The evaluation is used to 
identify areas for teacher imprbvemeht and for dismissal decisions. The 
district does hot believe that test scores and merit plan results are directly 
relevant for these purposes. 

The state government initially provided technical assistance for the 
merit plan and helped the district obtain a Title III innovation program grant 
of §5^i3O0* The state university has subsequently provided assistance in 
developing and scoring tests* Because of the State's concerns about legal 
Issues with the bonuses, the school system introduced the special voluntary 
merit pay contracts for individual teachers to assure that teachers understand 
that the extra pay is based on student performance and is a dhe-tlme bonus. 

Evaluatlbh of the Plan 

The system's main purposes dLh Introducing the plan were to encourage bet- 
ter reteijitibn bf teachers and better achievements by students. At the school 
board's request » the schbbl admiaistratlon undertook an external evaluation 
during the rthird year. It found that student gain scores have gone up cohsld- 
eraJiy over those in the fall. The district's scores on the ACT (taken by all 
studp.nts whether or not they intend to enter higher education) indicated sig- 
nificant increases over the period 1979-83. 

However, the district has little test scbre data prior to the beginning 
of the plan, and thus improvements over the pre-plan peribds carinbt be demon- 
strated. For the ACT scores^ which are available for the peribds 1969 through 
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1978, there were considerably higher scdris during the earlier years with a 
substantial drop-off occurring in 1977-1979 (before the start of the plan)i 
The adtninistration does not know why these dropped but the scores started back' 
up beginning in April 1988, the first year of the plan. The ACT cdmpdsite 
score (17.2 in 1983) is not yet badk to Khe 1969-1976 average of 18*4. The 
population of Seilihg has been fairly stable; there is no indication of a 
major change in factors such as the socio-economic characteristics of 
students. 

there has been little teacher turnover since the plan began and there is 
a perception by the administracion that the district is able to hire quality 
teachers, but the district does not have comparable data from before the plan 
started. It recognizes that busine^^s conditions cbvad also have affected its 
ability to retain teachers i 

Though the adtninistration could not point to specific differences in 
teacher procedures since the plan began, it believes that teachers are now 

more goal-oriented and more jealous of class time. In some instances teachers 

/ __ - - . . _. _ 

have complained about activities that haVe taken away from the academic time 

and appear to be focusing on planning activities related to their specific 

teaching object ives„rather than unrelated activities. The superintendent 

believes strongly that the plan has led to improved student achievement. 

There has been little reaction from the parents, though they appear to 
like the philosophy of pay-f br-perf btlnance. 

Students were interviewed by an butside consultant the third year of the - 
plan. The majority of the students were either positive or neutral towards 
the plan, but a few suggested that some students controlled their answers ^n 
order to reward or penalize certain teachers. To protect the teacher against 
this, the current procedure is to disregard the lowest 10 percent of the 
scores bn a test. 
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The teachers inter^ewed at the time of the evaluation expressed mixed 
views with some feeling that it had helped them financiaiiy and had motivated 
them. A few teachers said that it had caused more work or that it hurt self- 
esteem. Then^ and in subsequent feedback from meoibers of the advisory 
committee^ teachers expressed concern over the adequacy of the tests i 

The major problem expressed by both admihistratdrs and teachers Is the 
weakness in the testings especially whether the standardized t^sts cover 
material that the teachers are supposed Co teach. The school district is 
attempting to develop more localiy applicable tests that do this better, 
including skills tests such as for auto repair and home econdmics courses. 

A particular concern of the school district has been the dramatic fall- 
off of scores in the fall in the foiiowiug year from the reading and math • 
scores in the spring of the previous year^ There is some concern that 
teachers may riot be eriCdutraging students to do well on the tests in the fall 
(the baseline for determitang the change scores for that school year). Though 
the teachers themselves administer the tests » there are usually two teachers 
that Procter each test^ reducing temptatldns td affect student performance. 

The cost of the merit pay plan for bdniises averaged $755 per teacher over 
aii teachers. In 4aditioii there have been admlriistrative costs of about 
$2, GOO per year for a computer programmer and clerical costs, plus an annual 
cost of about $5,000 for the testing services. This additional $7,000 per 
year contributes an extra $160 per teacher, for a total of approximately $915 
per teacher; 

Suggestion s to Qthers - 

The superintendent suggested the following: 
d Keep the system simple; 

d Base the system on products and outputs (not on inputs). 

ERIC ^ 
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Don't take doiiars away from salaries to pay for the bonuses * 
Make the system voluntary. 

Do not tie the system into the teacher evaluation process used for 
considering disiiiissais and teacher improvement. 

Merit pay should be awarded on an objective basis; the administrator 
should be taken out of a subjective evaluation role. 
Tests should match the objectives of courses i 

involve the faculty in decision making, at least in an advisory 
capacity i 

Make merit pay available to the all that qualify. 
Sources : 

1. Gerald Daugherty and Gladys Dronberger, "A Merit Pay Program That 
Works: The Seilliig, Oklahoma, Experience^" S£ectrum, Summer 1983, pp* 
3-10. 

2. Educational Research Service, ISc, "Merit Pay Plans for Teachers: 
Status and Descriptions," 1983, pp. 46-47. 

3. Telephone Interview with Gerald Daugherty^ Superintendent, Seiiing 
Public Schools, December 1983 i 

f . „ 
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fredyf frin/Easttown School District 
Berwyn, Fennsylvania 

the Schooi Dtstrict 

The Tredyf frin/Easttown schooi district consists of eight schools (five 
eiemehtaryj two junior high, and one senior high); There are 326 teachers for 
4^400 students. The teachers belong to the Tredyffrin Easttdwn Education 
Association which is part of the Pehhsylvahia State Education Assdciatibn, in 
turn affiliated with the National Education Assbciatibh (NEA). The union 
engages in collective bargaining over wages and salaries. Under the current 
3-year contract, teacher salaries range from $14^500 to $31,019 with ah 
average of $27,500 for teachers with bachelor's degrees. 

Descr iption of the Pl an^ 

Tredyf f rin/Easttdwti has had two separate incentive plans: (i) the 
Perfbrmance Increments Program and (ii) the Superior Service Program. Only 
the second one is now used. Each one is described below; 

The Perfbrmance iacreia&nts Progr^; From 1970-1981, Tredyf frin/Easttown 
used the Performance Increments prbgram, a system of monetary payments related 
to job performance. The program was part of the contract package negotiated 
by labor and management. Ixi 1981i the program came to an end, ^'negotiated out 
of the contract*" 

f _ 

According to the program^ each teacher was given ah ahhual perfbrmance 
rating or score. An amount equal to 2 percent of the salaries bf teachers, 
counselors, nurses^ and library personnel was set aside for the perfbrmaiice 
increments. This total dollar amount was divided by the total bf all perfbr- 
mance scores to determine, at the end of the school year^ a dollar value for 
each performance point. This dollar value per point was multiplied by each 3 
teacher's scbre to determine the amount of the teacher's merit payment. This 
amount was added tb the base salary without fanfare or publicity* 
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Scores ran froin 0 to 6di if a teacher scored less than some pre-speci- 
fied figure I no award would be giveni this "bottom-ievei" figure was changed 
from contract to contract. During the last contract (78-81), no "bottom" 
figure was designated , and almost everyone received some awards 

The merit payments were awarded in July of each year, a month after the 
school year ended i Ml employees in the bargaining unit — teachers, counse- 
lors, nurses, media specialists—from all schools were covered by this plan. 

the performance ratings upon which the payments depended were based on 
classroom observations by the building administrator. Two to six bbservatioh 
visits were cdnducted for each teacher, some announced and others 
unannounced. After the dbservation, the admins trator discussed the assessment 
with the teacher. Annual performance appraisals were also part of the basis 
upon which each teacher was rated. 

Teachers also generated their own target goals, but this was done inform- 
ally between the teacher and the principal. Goals were not necessarily 
quantified. Teachers also cdnducted self -appraisals i 

Performance criteria Included the following: 

li Planning (e.g. "plans for the use of a wide variety of teaching 
strategics"). 

2. Climate for tearning (e.g. "shows respect for students as 
individuais") i 

3. Directing tearning Activities (e*g. "is effective in directing 
student interaction") i 

4. Evaluation df Instruction (e.gi "develops student self-evalixatidn 
skills"). 

5. Professional Attitudes and Conduct (e.g. "is self-motivated"). 
Each of the above was rated 0-12. the scales were hot "anchored". 

Different evaluation forms were used fdr nurses, counselors, and library 
specialists. 

« 
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tredyffrin has a very "hombgenebus" student body according to our respon- 
dent, with 85 percent going on to college; therefore, no effort was made to 
consider pupil "difficulty" in rating teachers. Evaluation procedures were 
the same for all the schools* 



had this merit bonus plan since 1970. This plan was never part of contract 
negotiations. From 1970-1979, these merit payments were added to the base 
salary. Since January 1979^ they have been bonus payments. Teachers have to 
"apply" to the program. 

About 4-5 teachers ^of the 326J receive the SSP bonus each year (about 
1-2 percent). Currently the bonus cbnsists of $750 awarded each year for 
three consecutive years to eadh teacher deemed to be "superior." A quota of 
4-5 winners is imposed because of the effbrt required to evaluate applicants. 

As in the case of the older program, all teachers* cdunseldrs, nurses and 
others in the bargaining rait in all sdhbols are eligible for SSP payments. 

The assessment of teachers is similar to the assessments under tUe old 
program with the difference that the rating is conducted by a team of three 
persons instead of by the building adiinist^atbr only. The three persons oa 
the team are U) a central adxiitnlstratori (il) a building adminlstratdr^ and 
(ill) a peer (teacher) who has received the SSP in the past. No anchored 
scales a^e provided. However, the respondent stPted that the 3-persdn team 
made the assessments more "acceptable" to teachers than were the old 1-person 
assessments. 

SSP awards are made in November of the school year following the 
performance period and are announced publicly. About 6Z-7% of the total 
instruction staff have received the SSP award at one time or another. 




The Tredyffrin school district has 
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Evaiaatjfcon of the Plan 

No formal evalaatton has been conducted of either of the two plahs. The 
PerfortDance Increments Program was "negotiated" out of the contract in 1981. 
The union preferred across-the-board increases i The sabjectivity of the per- 
fdrmaace assessments was one of the reasons cited by the union for dropping 
the program. The other reason given was that the average teacher ratings 
reached about 51-52 out of a total of 60^ with the range of scores very smalli 
The result was that almost everyone received a merit payment > thereby distort- 
ing the program as a "merit" payment plan. No evidence was avai^ble on the 
validity and reiiabiitty of the procedures. No special administrative or 
other requirements were needed for the plan. No major impacts were reported 
by the school district. 

The Superior Service Program has not involved any significant administra- 
tive costs. The bonuses cost the school district about §12,060 per year 
(about $40 per teacher in the district )i The district plans to continue the 
program. No ma^or impacts were reported by the school districts 

Sources 

1. Educational Research, Service, "Me r^ Fay Plans for Teachers: Status 
and Descriptions^" 1983, pp. 84-88. 

2* Telephone Interview with Mr. John Beatty, Director of Personnel^ 
Tredyffrin/Easttdwn School District, Berwyti, Pennsylvatiia, December 
i983i 

« 

3i Various materials provided by the school district. 
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Brown Deer School Districc 
Brown Deer, Wisconsin 

The School District. 

The Brown Deer School District currently serves about 1,800 pupils; 
There are three schools : ah elementary school^ a middle school (4-8) ^ and a 
high school, the school district's 125 teachers are affiliated with the NEA 
and bargain collectively over wages and salaries. 

Descr Lp tion^qf _t.fae_ Piaa 

for Cue pas: eight years, since i973, tne "brown Deer Scrtool District has 
been using various target-setting techniques to help motivate and evaluate 
teachers. Before that time, the school district utilized a traditional 
checklist approach for assessing teacher perf ortnance . But in 1975, the 
district began to emphasize the specif icatidii and achievement of job 
targets. Target achievement has hot, however, been linked to salary. 

In recent years the targe t-setting process has been de -emphasized 
somewhat and coupled with other perfbrmahce appraisal techniques. At present, 
teachers in Brown Deer can choose to be evaluated under any one of three 
procedures: (i) a checklist, (25 a modified (shortened) checklist i or (3) an 
evaluation approach of their own design. About 2 percent currently choose to 
use the full checklist, 78 percent select the modified checklist^ and about 20 
percent design their own evaluation approach^ Appraisals employing checklists 
usually involve one or two classroom observations of the teacher during the 
school year. These observations (each of which involves pre- and post- 
dbservatibh cdhferehces) are conducted by the school principals As described 
later^ each of the three procedures involves the specification of targets^- 

Teachers wishing to design their own perfbrmahce evaluation approach can 
do so with the concurrence of their principal. Usually these evaluations are 
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based on the teacher's completion of special projects or the achievement of 
other practical objectives during the coming year. Examples include ah 
attempt to use student evaluations in appraising a teacher's perf brinahce ^ and 
the develbpmerit and testing of new teaching techniques (such a project could 
include an experimental trial of the technique and an assessment of the re- 
sults by another teacher). If the teacher and the principal cannot agree on a 
specific evaluation approach, the teacher's proposal is submitted to a panel 
of two teachers and an administrator who review and resolve the conflict. 

Each of the three performance appraisal approaches involves the specifi- 
cation of perfonnance targets. Thus, all teaching staff must set a few tar- 
gets Ccommonly referred to as "growth objectives") each year. If the teacher 
has chosen to be evaluated by checklist, the growth objectives tend to focus 
on the behaviors and characteristics included in that checklist . Such a focus 
is hot required^ however, and many targets involve tasks and objectives hot 
directly related to items on the checklist. Target-setting plays an especi- 
ally important role for teachers who design their own appraisal approach since 
the special projects undertaken by those teachers are usually defined in , terns 
of a set of performance targets. 

The performance objectives are drawn from several sources: the perfor- 
mance appraisal checklists and the specific behaviors addressed by those 
checklists, specific problems that have beeh found in connection with a 
teacher's performance, job descriptions, and special projects or activities 
that the teacher wants to undertake. The targets are also supposed to reflect 
current school and district objectives (copies of these are prominently 
posted). 

Most targets are designed to improve teacher performance and capabil-' 
ities. Thus^ a teacher's performance targets can focus on personal growth 
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(eigi the completion of speciaiized courses) as weii as instructional proce- 
dures and outcomes i An jexampie would be "to review the reading materials at 
the fourth ^rade level and determine the need to revise the curricuiumi" 
Another example involved two teachers who were cooperating in observing and 
providing feedback on each other's performance. The performance targets 
specified by these teachers focused on making the necessary bbseirvatibhs and 
providing the feedback required. 

Teachers meet with their supervisors in the Spring to assess the past 
year's performance and Co specify perfomiahce targets for the upcoming year. 
Each teacher sets at least one performance target , and many have several 
targets i Both the teacher and the supervisor (principal) can propose objec- 
tivesi However, if there are problems with a teacher's performance, the 
supervisor usually takes the initiative in proposing targets that focus on the 
problem areas. 

No formal effort is made to adjust for differences between teachers in 
course assignments or pupil difficulty. (The principals are supposed to take 
such factors into cbnsidetatibh when assessing teacher performances) However, 
administrators attempt tb enhance cbmparability between the schools by holding 
joint discussions of the perfbrmahce ratings with all three schbbl principals* 

The targets can be modified during the schbbl year if bbth the teacher 
and the teacher's supervisor agree tb the change. Such mbdif icatibhs are 
reported to be common. 

Feedback on teacher performance and target achievement is prbvided at 
least twice a year and in some cases as often as every mbhth. The principal 
also meets with each faculty member for a formal assessment of overall 
performance and target achievement at least once a year and sometimes as often 
as quarterly. Each principal is supposed to spend at least four hours per 
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year supervising the development of eacn teacher i Part of this time is 
devoted to a discussidri of target achievement i if the specified targets are 
hot being achieved, the principal probes for the reasons why and tries to 
identify appropriate types of remedial aciiohs. (On the average, only about 
two percent of* the staff is involved in such remediation in a given year.) 
Poor performance with respect to target achievement is likely to lead to the 
assignment of specific improvement goals for the next year. 

While the school district provides no f aCTal recognition or award for 
butstahdirig perf oraance with regard to target achievement, such performance 
car. serve as the basis for a recommendation by the principai to a local 
cbminittee which selects the recipients of three annual "excellence in 
teaching" awards. These $500 bonuses (one for an elementary school teacher, 
one for a middle school teacher, and brie' for a high school teacher) are 
sponsored by a local industrial firm- 
There is no formal appeal prbcess fbr persbris using the checklist or the 
modified checklist evaluation prbcedure^ unless a disciplinary paction is 
potentially involved. Teachers choosing, tb develbp their bW p^i:fbnnance 
appraisal method can, as noted previously ^ appeal tb a panel of two teachers 
and brie administrator to resolve conflicts between the teacher, and the 
principal over the evaluation procedure to be used. The results bf such ati 
evaluatibti, hbwever, cannot be appealed. 

The Teacher's Association did not participate in the develbpmerit bf the 
.briginal target-setting procedure in 1975. However, the administration wbrked 
closely with the Assbciatibri fbr eighteen months to develop the option 
allowing a teacher tb design his br her wn. appraisal approach. 
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Evaluation of the Plan 

Brown Deer's performance targeting program has hot been fdrmaily 
evaiuatedi Admihis trators are unsure of the impacts of the target-setting 
effort on students or on teachers^ although they feel that it has been re- 
sponsible for some beneficial changes in teaching practices and procedures. 
The effectiveness of the target-setting efforts has apparently varied, depend- 
ing on the principal involved. Some principals have reportedly been success- 
ful in utilizing the approach to identify and enhance the strengths of the 
teaching staff. Others, however, have tended co use the target-setting proce- 
dures for highlighting a teacher's weaknesses, a strategy which administrators 

c 

feel may be nbtivatibhally less effective. 

The recent Changes allowing teachers to design their own evaluation 
approach are believed by school administrators to have relieved some of the 
pressure on the target-setting process. The new appraisal ojption was one of 
several actions taken to help reverse an apparent decline in Brown Deer's 
target-setting programs One symptom of that decline was the fact that over 
the years the goals that were established by the district's teachers had be- 
come relatively insignificant and easily achieved. This was believed to be 
due in part to a growing p e rcept ion^ by the teachers that they would be 
penalized for not meeting their job targe tsi Ths teachers responded by 
specifying less significant targets and goals that were relatively easily 
achieved. 

To avoid the perceived threat that came. to be associated with the perfor 
*^ * 

mance targeting effort (a threat which was, in fact, unwarranted since , there 
were no sanctions for hot meeting one's targets), the performance targets were 
renamed "growth objectives" and linked with the three alternate appraisal 
techniques described previously. This de-emphasis of the performance targets 
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was coupled witn renewed assurances oy scnooi adminiscracorsi lmcil t.uex:e wej.e 
no penalties for being unable to achiiavia bhe's targets. MdMbverj teachers 
who still felt that such targets were unfair or dthervr'.se undesirable were 
given the option of developing alternate appraisal approaches. Admiriistratbrs 
feel that by providing such an option, they have eliminated much of the threat 
formerly associated with the use of job targets. (Note that despite the 
availability of this option, about 80 percent of Brown beer's teaching statf 
have opted for checklist evaluations and the associated performance targetsO 

Schbbi district administrators believe thau teachers generally have a 
gooa understanding of the overall e\aluacibh prbcedure, although some of the 
specifics (eigi the target-settirtg procedure) may hot be understood as well. 

Administrators report that the target-setting process has impbsed nb 
special demands in terms of data processing or other special operating needs. 

Suggestions to Others 

Schbbi administrators in Brown Deer suggest that perhaps the greatest 
stimulus to teacher performance is the provision of aon-monetary recognition — 
the knowledge that bthers know and care about the teacher's performance. The 
administrators feel that by providing teachers with an opportunity for devel- 
oping their own individual perfbrmarice appraisal procedures, they have under- 
scored the administration's cbmmitmeht tb recognizing and evaluating teachers 
as tndivtdnalig i Brown Deer schooJ adminis tratbrs suggest that more frequent 
observation of, and feedback on, teacher performance may in itself increase 
productivity by enhancing teacher recognitibn^ morale^ and the overall 
operating climates 
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ii Mi Donald Thomas, Performance Evaluattbh of Educatlbhal Persbtiriel ^ 
Phi Delta Kappa Educational Fbundatibh (Blbbmihgtbn^ Indiana i 1979) ^ 
ppi 36-37i 

2. Telephone interview with Mr. Kenneth Moe^ Superintendent of the Brown 
Deer School District, December 1983. 
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nyae rarx, iNew lOI^t^. 

The School District 

The Hyde Park Central School District serves approximately 4,200 
students • it consists of five elementary schools^ one ;3unibr high school, and 
a senior high schbblw Gurrently the district has about 250 teachers. They 
are represented by the Hyde Park Teachers Association^ ah NEA affiliate ^ and 
bargain collectively over wages and salaries i 

l>cS;:rirrior c: the Flan 

Hyde Park introduced a classical, non-monetary management-by-objectives 
(MBD) plan in 1972. It lasted until 1978 when a more limited application of 
target-setting was adopted. The latter program continues to the present time* 

Hyde Park's original MBD plan was initiated in the wake of increased de- 
mands for teacher accountability and the need to provide better supporting 
data for tenure recommendations. The school board instructed the administra- 
tion to develop a more systematic procedure for appraising teacher perfdr- 
mance. The plan was developed through the joint efforts of teachers and 
administrators. After reviewing a variety of performance appraisal prbce- 
dures^ the group decided on a version of MfiiDi The plan was pilot tested with 
60 teachers (about one-quarter of the staff) for the first five months of 
1973. At the same time, teacHers and supervisors were given extensive in- 
service training in performance appraisal. The training included discussion 
groups, role playing ^ and assistance from well-known MBO consultants* 

The school board established a number of district-wide performance goals 
on basic student skills. Input from the caiimuriity and from staff was used in 
preparing these goals. The district-wide goals provided a focus for teachers 
setting their own job objectives. 

21B 



HP - 2 



Hyde Park's initial MBO system involved five steps: 

1 • Idehtif icatibn of Performance Areas > Four general perf btmance areas 
were identified: ihstructibhal skills^ management ability^ 
professional responsibility^ and professional qualities • Each area 
was broken down into sub-areas which were further defined and 
illustrated with examples of appropriate behaviors For instance, the 
general area of "instructional skills" was broken into "planning and 
organization," "appropriateness of materials," "resourcefulness and 
" adaptability," "ability to motivate/' "use of resources," "ciassroon: 
techniques/' and "parent relationships." Among the desirablie 
behaviors listed under "ability to motivate" were: deals with each 
student according to his heeds; uses a variety of classroom 
activities; varies^ assignments according to student needs; etc. The 
teacher and the teacher's supervisor jointly reviewed these lists of 
behaviors and identified ah (uhspecif iedj number of specific job 
ofijectives on which to concentrate. Generally these involved areas 
of performance that needed strengthening ^ improving^ or maihtaihihg. 

2i Setting of Individual Qbjectivas_ > The teacher and the teacher's 
supervisor next determined the teacher^s objectives for the coming 
year. The objectives were supposed to be specific (and be 
quantitative where possible:) and to identify what data were needed to 
monitor progress and what would constitute meeting tfhe objectives 
Some examples : 

The teacher will cpncehtrate on individualizing the mathematics 
• program .l>y developing a diagnostic and prescriptive approach to 

" the skills areas taught^ having at least 70 percent of the 

students achieving at grade level or above oh the May 19 State 

Achievement test. 



The librarian will initiate the "Otie-Tb-One" reading program for 
students in grades 4-6. The number of books read by students 
shoold increase by ID percent over the previous yeari 

Although the objectives were expected to vary considerably from 
teacher to teacher, ail objectives were supposed to focus on one of 
three areas: student progress, teacher performance, or overall 
program attainment (district-wide objectives In some cases ^ 
building-wide objectives were also established for specific schools. 
3. Development of Work Plan. After the objectives were identified, a 
detailed plan for achieving them was prepared. This also involved 
determination of the evidence that would, at the end of the appraisal 
period, indicate the extent to which the targets had been achieved i 
Procedures for collecting the necessary data were also agreed oni 
4i Monttortnp of Performance . Provision was made for regularly monitor- 
ing the performance of each teacher and for providing feedback on 
target achievement i (No general requirements oh the frequency of 
that feedback were specified.) 
5. Cottferencitig an d Follgw-ttp « At the end of the evaluation period ^ a 
conference was held between the supervisor and the teacher. Perfor- 
mance during the period was reviewed, and possible job objectives for 
the next round of appraisals were discussed* 
The fbregbihg procedures adhered closely to the traditionai MBO process 
recommended by private sector consultants. 

As time went by^ hovTever^ the number and quality of the objectives being 
prepared deelined, according to district officials. A more limited 
application of target-setting was adopted by the school district beginning 
with the 1978-79 school year. 
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In the current program the evaluation procedure now relies primariiy on 
supervisor observations. Targe t-aettlng Is used In only a limited way^ 
primarily as an adjunct to the teacher evaluation system, and then only for 
non-tenured staff and temired staff with performance problems. 

An appraisal project is required to ^be completed by each non-tenured 
teacher during the first three years on the job. The appraisal project is 
jointly defined by the teacher and the teacher's supervisor. It includes a 
set of objectives, criteria and procedures for assessing target achievement ^ 
and a standard for identifying when the objective has been achieved. A 
standard format is utilized for preparing the target, and both teacher and 
supervisor sigh off oh it. Appraisal projects generally last from 6 to 12 
months and focus oh a single specific area. Only one such target is assigned 
to a teacher at any given time. 

in the case of tenured staffs appraisal projects are assigned only if 
there is concern with a teacher's performance. Such projects are usually re- 
medial in purpose and target specific areas of weakness (e.g. planning, ciass- 
rodm discipline, instructional skills, etc. J. The contract with the Hyde Park 
Teacher's Association requires that such a project be tindertaken prior to a 
dismissal action. (Mote that the entire teacher appraisal system is specified 
in the contract.) Appraisal projects can also be assigned to tenured staff at 
6 to 12 months and focus on a single specific area. Only one such target is 
assigned to a teacher at any given timei 

These target^ can be altered or adjusted as needed during the performance 
period with one exception^ the latter being projects that may lead to denial 
of tenure. Such targets cannot be altered during the performance period. 
Regular feedback on target achievement is provided during the performance 
oeriodi The freauency of such feedback depends on the type of project and 
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No award or other recognition is given for butstandlhg p^^^fo^^i^c^ with 
regard to these targets, with the possible exception (in the case of non- 
teriiired staff) of a letter of commendation for the teacher's personnel ftlei 
Very poor performance with regard to the assigned targets can of coarse result 
in dismissal or denial of tenure. 

EyaloatiorL of the Plan 

There was no formal evaluatibh of Hyde Park's MBO programs. District 
officials reported that student achieveTCnt scores rose during the first MBO 
effort (note that the improvement of such scores was a major District objec- 
tive at that time), liowever, as noted previously, the quality and number of 
objectives specified in connection with the MBO plan declined between 1972 and 
1978. One school district administrator reported that Hyde Park's inittai MBO 
program began to lose effectiveness after just a few yearsi 

Hyde Park's current, more limited application of targeting has been 
reported to be effective for those teachers to whom it has been applied, in 
particular^ the performance of teachers assigned appraisal projects has 
apparently improved in many cases. The program has also reportedly led to 
some valuable curriculum development efforts. Note that since the current 
program is limited to individuais facing potential dismissal or a tenure 
decision, there is a strong incentive for the targets to be taken seriously, 
this reportedly accounts in large part for the effectiveness of Hyde Park's 
current application of performance targets. 

As a way to address individdai teacher problems and achieve specific 
improvements in teacher performance, the limited use of perfbxrmance targets 
linked to specific appraisal projects appears to have been effective in Hyde 
Park. However^ school officials feel that there are no clear overall motiva- 
tional advahtaees to their present approach to targeting as compared with 



other forms of clinical 6i iomiative evaluations or a ^ie comprehensive HBO 
system. 

Suggestions to Oth&rs^ 

School administrators made the following suggestions for overcoming the 
problems identified and for ensuring the effectiveness of an MBO program for 
teachers, based on their experiences with Hyde Park's first MflO program (Gray 
and Bains, 1979): 



° e^?fJ^^°''^ should not overdo the "collegiai approach" in developing 
?"e s^rin^KSl" -^Jectives with teachers. Supervisors sEo2d 

tlrgetrre^JicfSfdistrictS^^^ - — with 

° ^° prepare a work plan indicating how the targets are 

to be achieved. At the same time, the supervisor should specify the 
assistance that the administration will provide to help thfteacher 

thelrihriH'K:"*' Administrator. Stressed^hat iKSly 

there should be no surprises during the performance period. 

° ^Jt^.^^^^^ ^^"^ ""^^ °* education Should establish and 

publicize <listrict-wide objectives using the same format as the 
objectives to be prepared by lower-level staff. 

o ^e absence of a System of financial penalties or rewards linked to 
target achievement was viewed by some as detrinihtal to the 
laotivational effectiveness of the MBO program. 

o It was stressed that a teacher*8 objectives should allow for the un- 

Sn^ jL r K?^''*^^°°^*' ^° ^"^^^^ ""^^ sttuations that 

will inevitably arise* in the absence of such allownces, Hhe ^ 
achievement of pre-planned objectives will too often be pushed aside 
by crises* 

On the other hand, Hyde Park school officials noted two potentially 
serious issues for which they could offer ho solutions: 

o Contractual specifications of the performance appraisal process re- 

'^''^ °^ i° Sy^e Park. In particular, 

Hyde Park s contract with the Teacher's Association specified th^ 

utSSed! ^PP"^^^^^* "^^'^ the types of evaluations to he 

b Another p?5blem~one perhaps int.eren:. in developing an MBO progran 
for teachers (but not usually associated with private sector 
programs)-- was the large span of control characteristic of scho5ii. 



This Qseant that s single principal or assistant principal had to 
evaluate a large number of teaching staff. Such a situation may lead 

b ineffective evaluations and to reduced motivational effectiveness 

. the program. 



Sources : 

!• Frank Gray and Margaret L. Burhs^ "Does 'Management by Objectives' 
Work in Education?" Educational Leadership (March 19795^ pp. 414- 
417. 

1. "Teacher Appraisal Manual i" Hyde Park Central School District ^Hyde 
Park, New York, September 1975) i 

3. Telephone Interview vd.th l^loyd Jaeger, Assistant Superintendent for 
Iris t ruction, Hyde Park Central School District, December 1983 i 
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other forms of clinical or summative evaliiat-f rSne or- o ^^^^ - ^ : 

oiAiumai-Ave evaxuacions or a more comprehensive HBO 

systemi 

^Q^g^tibns to Others 

School administrators made the following saggestions for overcoming the 
problems identified and for .n^uring the effectiveness of an MBO program for 
teachers, based on their experiences with Hyde Park's first MBO program (Gray 
and Burns, 1979): 

° o^SJ^^°" should not overdo the "cdllegial approach" in developing 

tKf s^rinitlSl' objective, ^th teachers'. Supervisor' .^oSd 

tarIet^reflict£^Lf^.^^^^ especially in connection with 

i-argecB reriecting district-wide priorities. 

° tl to prepare a work plan iSdicating how the targets are 

to be achieved. At the same time, the sapervisof should spicify t&e 
assistance that the administration will provide to help the teacher 

SerJlhoal^lr" Str...id'that ideliiy 

there should be no surprises during the performance period. 

It was emphasized that the board of education should establish and 
publicize district-Wide objectives using the same format as the 
objectives to be prepared by lower-level staff. 

^y^""Hf °^ ^ ^^^""^ °^ financial penalties or rewards linked to 
target achievement was viewed by some as detrimental to the 
motivatlooal effectiveness of the MBO program. 

fL^Lf K^^i^^ f*^' ^ ior the un- 

wiii iSxH^.K?^'' f "^^'^ " ^^"^^^ ""^^ <^'i«is situations that 

will inevitably arise. In the absence of such allowances, the 

ly ctltitT ^"-P^^'^^^^ objectives will too often be pushed aside 

en the other hand, Hyde Park school officials noted two potentially 
serious issues for which they could offer no solutions: 

SrS?'""^^ ^^^^1""^^°''^ °^ the performance appraisal process re- 
portedly^constrained the use of MBO in Hyde ParE. In partlcullr 
Hyde Park's contract With the Teacher's Association spKlfiS thi 
Utilized"! ^PP'^^*^^^"' ""^^"-^ ^^'^ ^fie types of evaluations to be 

toother prdblei-~oSe perhaps inherent in developing an MBO program 
for teachers (but not usually associated with private sector 
programs)-- was the large span of control characteristic of schools 
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This neant that a single principai or assistant principal had to 
evaluate a large number of teaching staff* Such a situation may lead 
to Ineffective evaluations and to reduced mbtlvatldnal effectiveness 
of the program* 



Sources ; 

1. Frank Gray and Margaret L» Burns^/'Does 'Hahagemeht by Objectives' 
Work In Education?" Educatibaai Leadership^ (March 1979), ppi 414- 
417. 

2* ••Teacher Appraisal Manaai," ^de Park eentral School District (Hyde 
Park, New York, September 1975)i 

3. Telephone Interview with Lloyd Jaeger, Assistant Superintendent for 
last ruction, Hyde Park Central School District, December 1983. 
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RidgeWddd Schpdl District 
Rldgewoddi New Jersey 

The School District 

the Ridgewbbd School District has 4»996 students, 360 teachers and ten 
schools: seven elemeDtary schools, two junior highs (7-*9), and a senior 
htghi Most Ridgewood teachers belong to the Ridgewood Education Association, 
an affiliate of the New Jersey Education Association and the NEA. They 
bargain collectively dver wages and salaries. 

Descriptidh df the Plan 

Rldgewodd uses a versidn of performance by objectives (PBO) in cdnjuhc- 
tibh with its annual performance appraisals of teachers. Success in achieving 
one's objectives is hot linked to monetary rewards. 

The PBO prdgrarn evolved from teacher appraisal procedures initiated by 
the school district in 1964. These procedures were reviewed and revised five 
times in the succeeding 20 years ^ usually to refine and clarify the appraisal 
procedures or criteria usedi The most recent revision of Ridgewbbd's teacher 
appraisal plan occurred in i979i The primary impetus for these mbdif icatibhs 
was a change in New Jersey laws affecting teacher evaluations and the desire 
of the Rldgewodd School Board to make their appraisal procedure consistent 
with the new legal requirements. One result was greater emphasis upon annual 
goal-setting by teaching staff. 

The statute that stimulated these changes was Section 6: 3-1 • 21 of the New 
Jersey Administrative Code. This statute imposes a number df requirements du 
the evaluatibh bf teaching persbhhel^ Ihcludlhg a stipulatidn that the teacher 
evaluatibh prbcess include both ah individual prdf essibiial isiprdvement plan, 
and an examihatioh bf ihdicatbrs bf pupil progress to assess the individual 
teacher's performance. 

22. 



. Rldgewbdd's prior perf bnnaiice appraisal procedure had already addressed ^ 



achievement and the identification of needed improvements. The pre-1979 
teacher evaluation procedure involved subjective supervisory assessments of 
numerous aspects of instructional competence, professional growth and 
relationships, and personal characteristics related to teaching. There was 
also some (limited) goal-setting. 

Efforts to matce the performance appraisal procedure more compatible with 
the hew state code led to a greater emphasis oh target-settihg. ahd to the 
ihtrbductibh of new sectibhs bh the appraisal form dealing specifically with 
the individual's professional imprbvemeht plah ahd relevant ihdicatbrs of 
pupil progress and growth. The phrasing of the statute emphasized the Impbr-^ 
tance of setting objectives i For instance, the statute requires "review of 
teaching staff members' progress toward the objectives of the individual 
professional improvement plans developed at the previous annual conference" 
and "review of available indicators of pupil progress and growth toward the 
program objectives." 

Rldgewobd^s target-setting process should be viewed as part of the school 
district's teadher appraisal system. The apjpraisal system has the fbllbwing 
elemehts: 

!• Regular Testing of Student Achievement . All students in the Ridge- 



at least indirectly^ some of these requirements — e.g. the monitoring of pupil 



wobd schbbl system are giveh stahdardized tests bh ah ahhual basis. 



The Calif orhia achievemeht test series is used. 



Results are prbvided 



to teachers by the end of the school year anci are carefully examihed 



to identify their implications for future instructional efforts. 




As required by New Jersey's Administrative 



Code, the Rldgewood School District conducts in-class observations of 
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all teaching staff. Non-tenured staff are observed at least three 
times per year by th^echboi principal and/or a team of eyaluators* 
Tenured staff receive at least one classroom observation per yeari 
Each classroom observation is followed- by a conference. In addition, 
ail tenured and non-tenured staff recei-^'e a final summary evaluatibtl 
conference at the end of the year* the ih-class observations and the 
associated conferences address teachers' success in achieving their 
objectives with regard to professional development and pupil 
progress, as well as other factors rated oh the summary evaluation 
form (see below). 

3. Summary Evaluation Form . A written "Summary Evaluation of Teacher 
Competence" is completed at the end of the year for each member of 
the teaching staff. Tenured personnel are rated on three major 
areas: ihstructibhal competence, the teacher as a professional staff 
member, and personal characteristics as they relate to teaching (such 
as conscientiousness, moral integrity, maturity, resourcefulness, 
adaptability, and sense of humor)* Several attributes are described 
under each of these areas and rated as satisfactory or 
unsatisfactory. (For example ^ attributes rated under instructional 
competence include careful planning and sound preparation, recogni- 
tion of and provision for the individual ^ students are interested and 
stimulated by the teacher, teacher has a command of subject matter, 
effective methods are employed, teacher provides for bhgding evalua- 
tibn bf the achievement of pupils, etci) 
the summary evaluation fbr both tenured and non-tenured staff includes 
(1) a discussion of indicatbrs of pupil prdgtess and growth, and (2) an indi- 
vidual professional improvemeht plan. These two areas, along with various 
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school district and school building objectives, generally constitute the focus 
for teacher gdal^settihg efforts in Ridgevbbd* 

Th<2 goal-setting process begins in September of each year^ when each 
teacher meets with the teacher's supervisor. They jbihtly analyze test 
results for the preceding year,* assess their satisf actoriness and the 
improvetflents needed, and set objectives for the coming year^ in most cases no 
more than five objectives are specifiedo While most of the objectives are 
tailored to the individual teacher, a few may represent group objectives 
reflecting district or building priorities^ 

Objectives emerging from the review of the test results generally do not 
target specific changes in test scores. Instead, such objectives tend to 
address in general terms the areas needing attention. An example might be 
"improve the teaching of spelling." 

Many of the teachers^ objectives are drawn from the professional Improve- 
ment plani Such goals can focus oh the correction of individual deficiencies 
as well as the continuation of professional growth. Examples of teacher 
objectives on professional improvement include: 

o Moderate expectations of what teacher and students can accomplish in 
one year of learning history. 

b Make increased provision for differences in learning abilities among 
students. 

Another third important source of teacher objectives is the set of" 

district and building gbals established within the school systems The school 

district usually establishes 4-6 very general objectives, along with a number 

bf related target areas. While these are revised on an annual basis, for the 

most part they tend tb remain stable fbr a number of years. Examples of 

district-wide bbjiactives Include the fblldwing: 

b Upgrade computer skills 
o Upgrade writing skills 

o 
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b Cbhtihue to refine the use of piipil progress iridicatdrs to improve 

instruction _ 
b Gbntinue tb mbhitbr the affirmative actibh plan 

in addition to these district-wide bbjectives^ individual principals can 

establish specific building-wide goals for their own schbbl# 

All teachers are subject to the annual goal ^setting process. Teacher 
evaluation procedures in Rldgewood have been negotiated with the Teacher's 
Assdciatibn and are spelled out in detail in the school district's contract 
with the assbciatidn. The cdntract even includes excerpts from the relevant 
New Jersey Administrative Cddes cdvering performance evaluation of teachers i 

However I despite the precise specif icatidh df evaluatidri prdcedures in 
the cbhtract dbcumeht^ the target-setting elements df the evaluation process 
are hbt specifically mehtibhed. Thus^ many df the details assdciated with 
implementing the target-setting prbcedure are left up td the individual 
supervisori For instance, while the teacher may suggest many of the targets 
for the upcoming year, the supervisbr will usually take the initiative in 
proposing very specific performance improvement targets if there is concern 
over the adequacy of the teacher's perf orxaancei A teacher's flexibility in 
setting performance targets is also constrained by the need tb reflect schbbl 
and district dbjectives in the individual targets for the yeari Targets can 
be altered dr adjusted during the perf ortsiance period if both the teacher and 
supervisor agree td the change. 

Teachers are provided feedback, on target achievement through the in-ciass 
observations and conferences scheduled throughdut the year (at least one p^r 
year fbr tenured staffs three times per year fdr ndn-tenured staff )i 
Additibnal feedback is prbvided during the final summary evaluatidn conference 
chat takes place at the end bf the schbbl year. 
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The targets do noif however^ generally specify the kinds of data and data 
sources that wlii be ased to assess target achievement* Information on a 
teacher's progress is drawn from a number of soarces: test scores, teacher 
obseirvattonSy etc; 

There Is no formal procedure for adjusting the targets to account for 
differences in student difficulty or other special circumstances associated 
with a given teacher or school* However , such adjustments are addressed » In 
party by the emphasis given in the evaluation process to differentiated 
instruction and tne importance of recognizing and providing for individual 
differences among students* . 

Despite the emphasis on annual target-setting and the monitoring of tar-* 
get achievenient, the teacher ^s performance with regard to the objectives does 
not play a xaajor role in the overall summary evaluation of teacher perfor-- 
mance. The evaluation form does not provide a specific place for assessing 
target achievement, although a few elements on the form are indirectly related 
to target achievement* Examples include "sets long-range goals which are 
clearly recognizable by all" (under Che topic careful planning and sound prep**' 
aratidn), "carries out the individual prbfessibnal imprdvetnent plan coopera- 
tively developed with supervisor" (under teacher cbntihues to growth prbfes- 
sibhally)^ and "contributes to the efforts to accomplish system-wide goals 
(e*g. academic Imprbvemeht projects) and specific objectives of the schobls" 
(under recognizes and fulfiils total professibhal responsibilities)* Thus^ 
target achievement constitutes less than 15 percent of a hdh-tehured teacher^s 
performance evaluation; it could conceivably 1>e ignored entirely in assessing 
the performance of tenured staff* Ridgewood administrators note, however, 
that performance with regard to target achievement is often described in the 
"comments" sections that accompany each of the individual performance rating 
elements* 
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Outstanding performance with regard to target achievement is rewarded 
only by a notation in the teacher's personnel file, (the Ridgewood School 
System does provide an outstanding service award for one teacher per year. 
This award Is rotated annually between elementary and secondary school teach- 
ers* Target achievement can, of course , be ati elen^tit in the recdmmendatidn 
of a teacher for such ati award*) Very poor overall perf brmance with regard to 
the summary evaluation can serve as the basis for denial of tenure ot^ for 
tenured staffs the withholding of a perf bnnance increment* However i as ex-- 
plained previously, target achievement is only one of a number of factors 
considered in coming up with a teacher's summary evaluation* 

teaching staff were involved in the development of Ridgewood 's original 
performance evaluation instrument in 1964 and in all subsequent revisions, in- 
cluding the 1979 changes that led to increased emphasis on objective setting. 
While the teachers involved in the development process were union members^ the 
teachers' assdciatidn did ndt have a fortnal role in the design of the 
appraisal procedure* Nevertheless, the performance appraisal procedure is 
incorporated in the language of the teachers' contracts (the school district 
negotiated with the teachers' associatidn over the actual evaluatldn forms 
used*) 

The 1979 revisions to Etldgewobd's teacher appraisal prdcess were devel- 
oped over the period of one school year* All revisions were reviewed in 
faculty meetings before final approval* Although the new e^luatidti approach 
and the corresponding performahce targets emphasized increased attention to 
pupil achievements^ there was no heed for special testing of students since 
the school system was already administering standardized achievement tests to' 
all students on an annual basis* 
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and meeting with the Individual teacher* However » such evaluative meetings 
were part of the overall appraisal process and hot uhlqiie to the use of 
performance targets. 

School administrators report that no major problems have 5een encountered 
in connection with the targets-setting process* They note^ however^ that 
teachers may have lost some flexibility with regard to curricuiam development 
and experimentation because of the inclusion of district--wide and school 
building objectives. School administrators feel that the presence of such 
objectives may make teachers somewhat more conformist and less inclined to be 
creative In the areas addressed by the distrlct-wlde objectives* 

On the other hand, school admlhlstratdrs believe that the target-setting 
process has been effective In motivating teaching persdnnel. It has also 
helped provide cdhtlhulty of direction from one year to the hext^ a result 
attributed In large part to the development and Ihcorporatlbh of dlstrlct^-wlde 
and schbbl-^lde objectives. Oh balance » school admlhlstratdrs see no major 
disadvantages to the target-^settlng effort. 

Sugges tlons- -tb-^hera 

School administrators in Ridgewood recommend that other districts cbntem- 
plating a performance targeting effort start slowly and work up gradually to- 
wards fall-scale use of objectives. They emphasize that such programs should 
be simple and should be developed in close cooperation with professional 
staff. Finally, they underline the importance of school board participation 
through the development of realistic, understandable district-wide objectives. 

SQiirg&fi 

1. "Goals of the Ridgewddd Public Schddls," Board of Education (Ridge- . 
wood. New Jersey, June 1977). 
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2. "Board iSoais— 1980-81 and 1981-82," "District Gbals~1982-83," arid 
"1983-84 District Goals," Ridgewood Public Schools (Ridgewobdi New 
Jersey) • 

3. "Evaluation of the teacher as instructor, Staff Member, Per son i" 
Guidebbdlt, Ridgewood Public Schools (Ridgewood, New Jersey ^ 1979) * 

4« "Summary Evaluation of Teacher Competence," tenured and non-tenured 
versions i Ridgewood Public Schools (Ridgewood^ New Jersey.) . 

5. Telephone Interview with Mr. Robert Sullivan, Director of Personnel, 
Ridgewood Public Schools^ December 1983. 
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Salt take City School District 
Salt Lake City, Btah 

The School District 

Salt Lake City has 37 schools ^ ihcludihg 27 elementary, 5 laiddle (7-8), 4 
regular senior high schools, and one high school for adults. There are 
approximately 1,000 teachers for about 25,000 pupils. The teachers have ah 
association affiliated with the NEA^ 

Description ^f-^h e Plan 

This is a noh-mbnetary , perf ormance-by-objectives piani it was begun in 
the 1973-74 school year and is Currently in its 11th yeari 

The plan uses a pyramidal set of objectives. The school board sets gen- 
eral objectives for the school district as a whole. ^Recent objectives have 
included the initiation of cbst^aving approaches and school learning 
achievement gbals^ though stated in general fashion. Each school has its own 
school community council that annually sets general school objectives, such as 
improving school discipline and possibly student learning objectives. The 
council members include the principal and a minlmtim o^ eight parents. The 
councils also identify procedures for determining the degree to which those 
objectives are achieved. The k-8 schools also have a peicmanent set of minimum 
basic skill objectives covering most subjects* School community councils 
could include objectives aimed at improvements above those minimum levels. 

in the fall, each teacher identifies a set of the objectives ^ putting 
them on an "accountability fortni" These are reviewed with the principal 
before being accepted by the latter. These teacher objectives cover a wide 
range of topics with many being process, rather than direct student learning ^ 
objectives. At least to some extent, these objectives are intended to reflect 
the school and school district-wide objectives. For example, in 1982-83 to 
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meet a scKbbl district cost-savings objective, some teachers included an 
objective "to cover each other's classes when a teacher is absent in order to 
reduce the number of substitutes." 

the principal in January prepares a written progress repor t _w hich goes 
through the administration to the school board 5 at the same time, the prin- 
cipal may informally review the progress of individual teachers. The prin- 
cipal meets with each teacher near the end of the year to discuss the extent 
of achievement of the teacher's objectives; Teachers indicate on the account- 
ability form their perception and evidence of the level of achievement. The 
orincipals then add their assessment. 

The completed accountability form is kept in the teacher's file for 1-3 
years; The results are hot used for cdmpensatidn purposes i 

There is no additibhal teacher evaluation process. Employees who are not 
making a satisfactory contribution are placed on "remediation." If remed- 
iation is not accomplished within 30 days^ a fbmial remediation team is 
established consisting of a principal, a learning speciaUst, and two teacher 
colleagues. The teams work ^th the person for a S^bhth period. At the end 
of that period a recoduaendation is made to terminate the teacher or to destroy 
all records if remediation has been successfully achieved. 

niere is a review/grievance procedure, especially fbr the remediatidti 
elements. 

For the 1983-84 school year a special ad hoc bonus plan based on 
district-wide cost savings was adopted i No general raises had been apprbpri- 
ated by the state legislature, instead, the school district decided to pro- 
vide an across-the-board three percent bonus paynent if substantial savings 
were achieved; By early December the district felt that enough savings had 
been achieved and awarded the three percent bonus. This bonus is indepehdeht 
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of the perf ormance-by-6bjectives plan^ except that cbfit-^savirig objectives were 
included in some school and teacher objectives. 

Evaluatibh of the Plan 

There has been no formal evaluation of the performance-by-objectives 
plan. The district administration indicates^ however, that the district was 
below the national norm before the plan began. The district currently aver-^ 
ages two years above the national norm. The school systemi however » has not 
formally tracked such figures as changes in test scores over time. The cur- 
rent administrator of the plan attributes this improvement to the cbmbihatidn 
of the accountability program and the associated focus bh minimum basic 
skills, and the remediation process. He feels that the system, by 
establishing bbjectives to achieve basic skill levels and retaining only those 
teachers suited to the educational setting, contributed significantly to the 
improvement. The district has a larger prdpbrtion of terminations of teachers 
as unsatisfactory than any school district in the state and possibly in the 
country i This is taken as evidence of having achieved its goal of retaining 
only effective teachers. 

Though initiaily the plan needed substantial selling to the 'teachers, the 
school administrators persisted In the iiiid-70'8. Currently the admins tration 
feels there is only quite minor negative feeling towards to ^ae plan. Parents 
appear to be supportive; their role as part of the schobl community councils 
means that. they have a larger say in school activities. 

The primary cost of the plan has been its time^onsumihg nature, required 
because of the involvement of so many people in the detemihatibh of bbjec^ 
tives and subsequently assessing performance i There is also additibhal 
recordkeeping required for the plan. However, no cost data are available. 

23S 



SLC - 4 



Sources 



ii Mi Donaid Thontas^ Performance Evaiaatlon of Edacatlbnai Personnel, 
Phi: Delta Kappa Educational Foundation, Bloomington; Indiana^ 1979^ 
ppi 33-36i 

2« Teiephone interview with Dr» Stanley Morgan, Administrator for 

Educational Accountability, Salt Lake City School District, Salt Lake 
City, Utah, December 1983. 



3. Various materials provided by the school district. 
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Santa Clara Unified School District 
Santa clara, California 



The School District 

The Santa Clara Unified School District consists of 15 elementary^ two 
junior high (7-9), and two senior high schools. There are approximately 
12^566 paptis and 65t^ teachers. Although a few teachers reportedly belong to 
the AFT, most are members of the United teachers of Santa Clara, an affiliate 
of the California teacher's Association and the NEA. Santa Clara teachers 
bargain collectively over wages and salaries. 

bescrlptibn of the Plan 

Since 1972, the Santa Clara Unified School District has been using a 
(nbn-mbnetary) version of performance by objectives (PBO). The program was 
originally implemented by school administrators to help improve accouhta-* 
bility. Ah additional stimulus was provided by the passage of California's 
Stull--R6dda Prof essiohal Competency Act^ which stipulated (among other things) 
that all certified teachers most receive periodic perfdrmahce appraisals* 
teachers and administrators In Santa Clara viewed PBO as ah attractive strat-* 
egy for achieving the t^h goals of regular perfdrmahce appraisal and Ih- i 
creased accoantability i (Prior to Che use of no^^ Santa Clara employed a 

checklist with subjective supervisor ratings of items such as classroom cli* . 

' '. 

mate, teacher dress, etci) 

The program currently operates in much the same way as it was originally 

dejsigned. At the beginning of each school year all teachers receive an evalu- 

atidn packet that includes the school plan (building objectives), teachers 

then specify their owti objectives for the coming year^ including specific 

targets ("success criteria") and suitable measures for assessing the degree to 

which those targets have been attained, the objectives usually address 
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student progress In various areas of study (the dtscrict has eccabiished stan^ 
dards for progress In each academic area and techniques for assessing that 
progress) » as well as classroom control, maintenance of the educational envi-* 
rdnmenty etc* The teacher perf drmance targets for assessing student achieve-* 
meht in each area of study are supposed to be the minimum standards of perform 
mance expected for individual students. The support services heeded from the 
school district to facilitate the attainment of the given objectives are also 
to be specified. 

Our respondent reported the following escamples to be typical of the 
objectives that might be specified by an elementary school teacher: 

i 

o My students will be introduced to long division using the approved 
curriculum, and 90 percent will be able to pass an exam in long 
division. 

d My students will write cdmpdsitidns dnce a week emphasizing good 
sentence structure. 

d We will have a science unit dn the Santa Clara Valley, and students 
will be able td identify area birds. 

d I will introduce a sdcial sciences unit dn ancient histdry, and each 
student will prepare a report on a specific country. 

Teachers are usually expected to cbhcehtrate bh about six objectives 
(considerably fewer than the number of objectives specified by teachers when 
the program began). A teacher's perfbrmahce targets can include personal 
develbpment goals as well as bbjectives focusing oh teaching outcomes. 

The objectives prbpbsed are alsb suppbsed tb relate tb and support the 
objectives of the district and bf the school. Althbugh copies of district 
goals are not included in the individual evaluation packets, they are pbsted 
in each buildingi District-wide objectives are generally develbped fbr a 
three-year period. Examples of such objectives (which tend to be very gen- 
eral) have included enhancement of reading scores, more homework, and better 
discipline; Building dbjectives are developed for each school by the building 




principal i These are designed to support the district objectives and to re- 
flect specific baiiding priorities i the building bbjectives tend to be more 
specific than the district objectivesi For instance^ at the primary levels 
there might be a building objective on raising the standardized test scores in 
mathematics or language (e.g. if warranted by the previous year's test 
results). However^ school administratbrs report that building objectives 
involving teaching outcomes are hot usually specified in terms of specific 
improvements in stahdarized test scores. 

Secondary schools in Santa Clara currently utilize a program of "guaran- 
teed instructional services" which specified the development of specific 
skills at specific points in a student's career. Successful teaching of these 
skills frequently constitates the basis for the goals or bbjectives specified 
by secondary school teachers^ 

All teachers are expected to draft the objectives themselves and meet 
with their building principal early in November to discuss and agree on the 
goals for the year. When agreement has been reached^ the teacher and the 
principal both sign off on the targets. The objectives mast also be reviewed 
and formally approved by the site administrator acting for the school dis- 
trict. 

During the year^ feedback da target achievement is provided by the school 
principals. The frequency of feedback depends on the principal; while some 
principals reportedly provide feedback oh a weekly basis^ in most inst&nces 
the feedback is less often. (The official guidelines for the program do not 
prescribe a specific feedback frequency.) School administratbrs report that 
feedback on target achievement has become much less frequent since the PBO 
program was first implemented in 1972. 
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It is possible for a teacher's objeGtives to be revised daring the coarse 
of the year with the concurrence of all parties. This might occurs for 
example, if there is a change in teaching assignment or in the mix of students 
taught. However, Sarita Clara school district officials could lidt recall an 
instance where such changes have been necessary* 

Target achievement is evaluated in March and April (final evaluations of 
all tenared employees are dae in mid-April). Eairlier interim evaluations are 
required of non-tenured staff and teachers whose performance has been below 
standard. 

Each teacher meets with the building principal to discuss target achieve- 
ment, reasons for under-achievement , and the improvements needed. The princi- 
pal's assessment of target achievement is based on classroom observations » 
conferences with the teacher, and aiiy other available data; A formal overall 
evaluation report covering seven specific areas and roles (e^^^ the teacher as 
a hutDan resource i as an assessor of student needs^ as a coomunicatdr In the 
educatibhal process^ as an evaluator of student progress and ihstructibnal 
purposes^ etc.) is also prepared. Perfdrmahce is/compared to predeterxalhed 
"standards" derived from role expectatibnSt pbsitibh descriptioasi schbol 
plans, student achievement goals and objectives^ scbbbl board policies^ and 
administrative regulations. (The relationship between a teachers'^ own per- 
formnce targets and the **standafd8" used to evaluate the teacher^s perfbr*^ 
m^nce is, however, left unspecified.) 

For each area or roie^ the supervisbr rates the teacher as "outstandings" 
"effective," or "improvement needed." if "butstahdihg" br ."imprbvemeht 
needed" is checked, the supervisor must provide an addendum with supporting 
details and, in the case of "improvement needed," a summary of the specific 
imjirovements required i T&rget achievement under the PBO system is often 



referenced in these sopporttng statements, there is^ however^ no place on the 
evaioatton form where the achievement of performance targets is spectftcaily 
reviewed. 

No tangible reward is given for outstanding performance with regard to 
target achievement* At mbst^ such perfbrxDaace reportedly earns a pat on the 
back and cohgra tula t ions from the principal* If perf drmance has been pbor^ 
the school district initiates a remediation program for the given teacher, 
this program involves^ the establishment of a team of resource people who work 
with the poorly performing teacher for &9 days. If the teacher^'s performance 
doe3 not improve by the end of that period, the teacher can be terminated. 

School administrators report that there have been no problems with regard 
to target comparability between teachers or between schools » even though there 
are no explicit procedures for taking account of differences in pupil odbc or 
course complexity. Any such adjustments are expected to be made, by the prin*- 
cipal as the latter sees fit*. (For example, if a teacher has several handi- 
capped students » the principal is expect:^d to decrease the class size accord*- 
ingly.) Nevertheless^ the district places considerable emphasis on having 
teachers tailor their educational techniques to meet the differing ..eeds of 
ihdiviual students. Teachers are urged to view this as a major district 
objective and to incorporate it in formulating their own objectives. 

There is ah appeal process for disagreements over a teacher^s perforii^nce 
targets for a given year. Thus^ the teacher has the bptidh bf gbihg tb the 
superintendent (or the superintendent's designee) in November tb resolve dis- 
agreements between the teacher and the building principal with regard tb the 
year's objectives^ 6to the other hand, the substance bf the ehd-bf-year evalu- 
ation cannot be appealed. 
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Santa Clara^s PBG program required approximately one year to deveiopi An 
extensive effort was made it that time to involve teaching staff In the design 
of the plan through a committee composed of admlnistracors and teachers from 
ail levels^ A survey of teacher evaluation needs was also conducted* 

In 1976» the cdmmittee was reconvened to review the P50 effort. The 
remediatibn program was added at that time. The PBO program is currently 
being reviewed again by a committee of teachers and administrators • 

The school district has provided some training to teachers concerning 
goal-^setting and the PBO approach. Woit of that training occurred when the 
plan was first implemented in 1972* More recently, there have been some 
refreshers on how to write goals and objectives^ School administrators in 
Santa Clara currently feel that n^^st teachers are preparing reasonably good 
goals and objectives i 

Evaluat i^n^^ the Plan 

There has been no formal evaluation of Santa C?ara's PBO effort. School 

- -- - - __ ^ 

administrators we spoke with could not judge the impact^ of the program on 

students or on teacher attitudes. They did) bbwever^ feel that the settlug of 

goals and objectives has been beneficial. In particular ^ it forces teachers 

to plan» and It follows tip on plans by mdnitbriilg their achievements with 

respect to that plan. This is believed to improve teaching practices and---' 

ulti2nately*<-teacher effectiveness^ although hb specific examples could he 

cited. The PBO prbgram was reported tb have had ho effect oh teacher reteh^ 

tibh br recruitment. 

Admihistratbrs report that teachers in the Santa Clara school district 

generally regard the P50 program as fair and objective* The program has not 

been the target of formal grievances^ (However^ there were some problems when 

one administrator failed to rate ^^ly^ of his teaching staff as "outstanding.*') 
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As lib ted earlier^ a comtriittee of teachers and administrators Is examining both 
the overall performance evaluation process as well as the Implications of 
California's new legislation concerning the establtsh^nt of master teacher 
programs, teachers have also been surveyed concerning their satisfaction with 
the performance appraisal (and PBO) processes. (No results frpizi that survey 
are available as yet.) 

For the most part^ Santa Clara's PBO program has received little atten*^ 
tibn from the public or the press. An exception ^ however^ is the remediation 
program for poorly performing teachers. This has received considerable hews^ 
paper publicity and Is reported to be supported by most parents. (On the 
other hand, some people have charged that the remediation program Is primarily 
a device to keep from firing teachers.) The names of the teachers receivlag 
remedial help are kept confidential^ 

There has been no assessment of the overall costs of Santa Clara's PBO 
program. No extra staff have been needed to operate the program* the remed±«* 
ation effort itself is estimated to cost about $5,000 per year (primarily In 
release time). In general, two or three teachers need remediation in a 
typical year. 

Overalip school dlstribt officials in Saiita clark feel strbngly that the 
PBO technique has been an effective mbtlvatdr of teaching personnel* Regulair 
gdal"*setting is credited with stimulating teachers tb perform above standard 
and with making thim more effective and mbre confident in executing their 
jbbs. « 

However^ school district administrators stress the Importance of malh"- 
talnlhg and periodically revitalizing a PBO effort such as theirs. They note 
that as the years go by^ teachers have tended tb develop few hew goals and 
objectives. A recent survey of Santa Clara teachers reportedly suggested that 
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awareness of the P50 program and the associated remedlacldn procedure had 
slippedi in addition^ some principals are reportedly doing a less respdhsible 
job of evaluating their staff. The problem lis eispecially acute for secondary 
school administrators. The latter reportedly spend less time observing teach- 
ers and are less likely to "bite the bullet" when there is a need for an 
unsatisfactory evaluation. 

The school district's current review of the PBO program is designed, in 
part^ to address these emerging problems. (Another task of the committee will 
be to ihcbrpbrate California's new master teacher program into the PBO 
process; it is expected that target achievement will be one of the criteria 
used for selecting master teachers.) A second approach being considered for 
revitalizing the PBO effort is to rotate building principals more 
frequently. (Building principals are currently rotated once every seven 
years, on the averages) it is expected that more frequent rotations could 
help counter the slow decline in the effectiveness of Santa Clara's PBO 
process. 

Source ; 

1. M. Donald Themafry Perfor manc of E d ucational Pe rsonnel^ 
Phi Delta Kappa Educational Foundation (Blddmington, Indiana, 1979), 
pp. 30^32. 

2. Telephone interview with Mr. Nick Gervase, Assistant Superintendent 
for Persohhel Services, Santa Clara Unified School District, December 
1983. 

3. Various materials received from the school district. 
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